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issues  in  Software:  A  Blue  Two  Visit 

Feasibility  Assessment 


Abstract 

Mhe  Software  Engineering  Institute  (SEI)  participated  in  a  series  of  fact-finding  meet¬ 
ings  sponsored  by  the  Air  Force  Coordinating  Office  for  Logistics  Research 
(AFGGLR)  to  gather  information  necessary  to  set  the  scope  for  and  to  implement 
one  or  more  Blue  Two  Visits  on  software.  The  purpose  of  a  typical  Blue  Two  Visit 
(BTV)  is  to  introduce  to  industry’s  top  design  engineers  and  program  managers  the 
day-to-day  constraints  Air  Force  maintainers  face  on  front-line  operations  bases. 
The  participants  experience  first-hand  the  effects  of  design  on  maintenance.  This 
exposure  has  been  significant  in  bridging  the  gap  between  DoD  and  industry  in  un¬ 
derstanding,  documenting,  and  supporting  Air  Force  weapon  system  requirements  to 
increase  combat  supportability. 


This  report  documents  discussions,  which  attempt  to  address  the  following  questions 
for  a  software-oriented  BTV: 

■  1 .  Do  software  maintainers  and  users  have  messages  for  software  designers 
and  programmers ?j 

2.  What  are  these  messages?  ' 

3,  How  can  these  be  best  communicated?  j 

4;  To  whom  should  these  messages  be  targeted?’  >  ^ 

5.  What  should  the  BTV  be  called?  !  < 

Participants  at  these  meetings  included  personnel  from  the  SEI,  AFCOLR,  Inter- 
Command  Electronic  Warfare  Management  Directorate  (ASD/RWA),  Ogden  Air 
Logistics  Center,  Warner  Robins  Air  Logistics  Center,  388th  Tactical  Fighter  Wing 
(TAC),  Detachment  1  of  the  49th  Test  Squadron,  Air  Force  Operational  Test  and 
Evaluation  Center  (AFOTEC)  B-1B  FOT  &  E  Test  Team,  96th  Bombardment  Wing 
(Heavy)  (SAC),  and  Headquarters,  Strategic  Air  Command.  The  issues  raised  by  the 
participants  in  the  various  discussions  did  not  necessarily  describe  all  concerns  af¬ 
fecting  software  development,  maintenance  and  enhancement  in  Air  Force  systems, 
but  did  provide  insights  into  significant  technical  and  management  considerations  of 
Air  Force  software  maintainers.  These  considerations  fell  primarily  within  six  areas: 
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1 .  weapon  system  orientation 

2.  acquisition  issues 

3.  post  deployment  software  support  issues 

4.  software  development  issues 

5.  supportability  and  testability  issues 

6.  personnel  management  issues 


i 
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1.  Introduction 


1.1.  Blue  Two  Visit  Program 

The  Blue  Two  Visit  (BTV)  program,  managed  by  the  Air  Force  Coordinating  Office  for  Logistics 
Research  (AFCOLR),  exposes  corporate  program  managers,  senior  design  engineers,  and  ap¬ 
propriate  Air  Force  systems  acquisition  personnel  to  "real  world"  operating  and  maintenance  pro¬ 
cedures  and  constraints.  The  program  does  this  by  facilitating  direct  communication  between 
these  key  decision  planners,  developers  and  the  day-to-day  mairTainers  of  current  Air  Force 
weapon  systems.  The  BTV  goal  is  to  allow  face-to-face  interaction  between  maintainers  and 
industry  designers  and  program  managers  in  order  to  influence  future  weapon  systems  and  sup¬ 
port  equipment  designs  to  be  more  reliable  and  easier  to  maintain.  By  coordinating  interaction 
between  the  technology  providers  and  the  technology  users  through  "hands-on"  exposure  within 
the  operating  environment,  the  BTV  program  has  influenced  designers  to  incorporate  reliability, 
maintainability,  and  supportability  features  into  future  weapon  systems. 

Originally  known  as  "Contractor  Visits  to  the  Field,"  the  BTV  program  began  in  December  1983, 
by  the  initiative  of  AFCCLR,  the  Joint  Advanced  Fighter  Engine  Program  Office,  and  LtGen  Leo 
Marquez,  then  Deputy  Chief  of  Staff,  Logistics  and  Engineering,  HQ  USAF.  In  September  1985, 
the  program  was  officially  redesignated  Blue  Two  Visit,  referring  to  the  Air  Force  "Two  Striper" 
(Airman)  maintainer.  Initial  field  visits  centered  on  a  small  class  of  known  operational  reliability 
and  maintainability  ills  specific  only  to  aircraft  systems  including  safety  wire,  corrosion,  hydraulic 
leaks,  non-interchangeable  components,  etc.,  and  addressed  common  service  problems  and 
methods  for  overcoming  faulty  design.  From  aircraft  systems,  the  program  expanded  significantly 
in  1986  to  include  new  acquisition  of  aircraft;  missiles;  and  command,  control,  communications, 
and  intelligence  systems  (C3I);  in  addition  to  existing  components  and  support  structures. 

The  BTV  forum  is  an  immersion  in  the  total  maintenance  experience  —  the  harsh  weather,  the 
long  hours,  the  lack  of  spare  parts  —  where  corporate  presidents  and  industry’s  aerospace  de¬ 
sign  engineers  can  be  "maintainers  for  a  day.”  During  visits,  participants  are  required  to  put  in 
long  hours  on  flight  lines,  often  "suiting  up"  in  chemical/biological/radiological  gear,  and  perform 
routine  maintenance  tasks  —  often  times  on  a  system  their  company  has  designed.  As  a  result 
of  the  experience,  the  Air  Force  is  witnessing  a  renewed  commitment  from  industry  in  meeting  Air 
Force  reliability  and  maintainability  goals. 

Since  conception,  the  field  visit  effort  has  matured  to  a  premier  Air  Force  program  for  identifying 
and  addressing  reliability  and  maintainability  concerns.  To  date,  there  have  been  28  Blue  Two 
Visits  with  535  contractors  (representing  165  companies),  249  DoD  and  3  academic  personnel 
participating.  The  28  BTVs  have  visited  78  units.  The  BTV  program  is  continuing  to  gain  recog¬ 
nition  and  support.  Planned  BTVs  include: 

•  Electronic  Warfare  Sep  87 

•  Electronics/Avionics  Nov  87 
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Aircraft  Support  (AAC) 

Jan  88 

Aircraft  Support  (PACAF) 

Feb  88 

Structures/Fasteners/Composites 

Apr  88 

Aircraft  Engines 

Jun  88 

Aircraft  Support  (USAFE) 

Aug  88 

Tactical  Weapons 

Oct  88 

In  addition,  major  USAF  commands  are  being  encouraged  to  set  up  their  own  BTV  programs. 
This  was  emphasized  by  General  Piotrowski,  Headquarters  USAF/CV,  in  his  Reliability  and  Main¬ 
tainability  Policy  Letter  #3:  Blue  Two  Visit  urging  that  BTV  program  become  a  mandatory  part  of 
the  acquisition  process.  Such  a  program  has  been  established  at  Hill  AFB  in  Ogden.  Ogden  Air 
Logistics  Center  (OO-ALC)  personnel  have  visited  the  shops  at  the  388th  Tactical  Fighter  Wing, 
and  the  Advanced  Tactical  Fighter  System  Program  Manager  at  the  Sacramento  Air  Logistics 
Center  has  brought  Advanced  Tactical  Fighter  (ATF)  contractor  teams  to  Hill  AFB  for  visits. 
Other  Air  Force  activities  have  established  similar  orientation-type  programs  under  various  titles 
which  parallel  BTV  objectives. 


1 .2.  Software  Blue  Two  Visit 

With  software  becoming  an  increasingly  essential  and  expensive  element  of  defense  systems’ 
acquisition  and  life  cycle  support  costs,  AFCOLR  was  directed  by  the  Assistant  Secretary  of  the 
Air  Force  for  Logistv's,  to  investigate  the  feasibility  of  one  or  more  Blue  Two  Visits  on  software. 
However,  because  of  software's  complex  but  sensitive  nature,  it  does  not  lend  itself  easily  to  the 
’•hands-on"  operating  and  maintenance  exposure  typical  of  a  hardware  BTV.  For  these  reasons, 
a  fact-finding  team  was  formed  to  attempt  to  answer  several  questions  in  order  to  properly  set  the 
scope  for  and  implement  a  successful  software-oriented  BTV.  These  questions  include: 

•  Do  software  maintained  and  users  have  messages  for  software  designers  and  pro¬ 
grammers? 

•  What  are  these  messages? 

•  How  can  these  be  best  communicated? 

•  To  whom  should  these  messages  be  targeted? 

•  What  should  the  BTV  be  called? 


These  questions  have  provided  a  framework  for  gathering  data  across  all  visited  Air  Force  main¬ 
tained,  including  organizational  level  maintained  in  the  units  at  the  front-line  operations  bases, 
depot-level  maintenance  personnel  at  the  Air  Logistics  Centers,  and  maintenance  evaluators  for 
new  weapons  systems. 
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1.3.  Feasibility  Assessment 

This  broad  approach  to  examining  Air  Force  software  maintenance  provides  the  greatest  potential 
for  obtaining  a  comprehensive  overview  ana  for  identifying  the  numerous  interrelated  messages 
that  should  be  communicated  to  defense  contractors  and  Air  Force  acquisition  personnel  in  a 
software-related  BTV.  This  report: 

•  Documents  the  activities  of  the  AFCOLR-sponsored  fact-finding  team. 

•  Summarizes  the  significant  technical  and  managerial  considerations  reported  by  Air 
Force  software  maintained. 

•  Provides  recommendations  to  Blue  Two  Visit  program  management  regarding  the 
feasibility  of  a  software-oriented  Blue  Two  Visit. 


CMU/SEI-87-TR-42 


5 


2.  Fact-finding  Visits 

The  fact-finding  team  assembled  by  AFCOLR  conducted  a  number  of  visits  to  investigate  the 
feasibility  and  projected  scope  of  a  software-oriented  BTV.  The  AFCOLR  Software  BTV  fact¬ 
finding  team  consisted  of  the  individuals  shown  in  Table  2-1 .  Appendix  B  to  this  report  lists  all 
participants  in  the  fact-finding  visits. 


AFCOLR  Software  BTV  Fact-finding  Team 

Capt.  Benita  L.  Gilliard 

Information  Systems  Branch  Chief, 
Plans  and  Programs  Division,  Air 
Force  Coordinating  Office  for  Logis¬ 
tics  Research 

AFCOLR/XRI  (HQ  USAF) 

Capt.  Daniel  R.  Bliss 

Data  Systems  Development  Of¬ 
ficer,  Logistics  Maintenance  Man¬ 
agement  Data  Branch,  Head¬ 
quarters,  Strategic  Air  Command 

HQ  SAC/LGMMD 

CMSgt.  Charles  M.  Worm 

Blue  Two  Visit  Program  Manager, 
Independent  Research  &  Develop¬ 
ment  Branch,  Maintenance  and  En¬ 
gineering  Division,  Air  Force  Coor¬ 
dinating  Office  for  Logistics  Re¬ 
search 

AFCOLR/MEI  (HQ  USAF) 

Landon  O.  Sager 

Logistics  Management  Specialist, 
Inter-Command  Electronic  Warfare 
Management  Directorate 

(ICEWMD),  Aeronautical  Systems 
Division 

ASD/RWA 

William  E.  Hefley 

Member  of  the  Technical  Staff, 
Technology  Transition 

Software  Engineering  Institute 

Table  2-1 :  AFCOLR  Fact-Finding  Team 


To  better  understand  the  environment  in  which  mission-critical  software  is  maintained,  key 
software-intensive  sites  were  chosen.  The  AFCOLR-sponsored  team  visited  several  Air  Force 
facilities  involved  in  software  acquisition,  support,  and  maintenance.  Each  facility  was  selected 
on  the  basis  of  its  intensive  involvement  in  aircraft  weapon  systems.  Major  organizations  con¬ 
tributing  to  the  success  of  this  software-oriented  BTV  feasibility  assessment  are  shown  in 
Figure  2-1. 

This  team  visited  the  Ogden  Air  Logistics  Center,  Hill  AFB,  UT,  and  the  96th  Bombardment  Wing, 
Dyess  AFB,  TX,  during  10-14  August  1987.  Additional  meetings  were  held  at  the  Warner  Robins 
Air  Logistics  Center,  Robins  AFB,  GA,  2-3  September  1987. 

At  each  of  these  meetings,  participants  were  briefed  on  the  BTV  program  and  the  AFCOLR  effort 
to  include  software  in  the  BTV  program.  Participants  then  discussed  their  view  of  software  within 
their  day-to-day  functions.  These  discussions  primarily  addressed  mission-critical  computer 
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2.1.  Hill  Air  Force  Base 

The  fact-finding  team  visited  the  Ogden  Air  Logistics  Center  (OO-ALC),  Hill  AFB,  UT,  on  11 
August  1987.  The  OO-ALC  is  responsible  for  depot-level  support  for  the  F-16  fighter  aircraft. 
The  host  for  this  meeting  was  Mr.  Rick  Holsman,  Branch  Chief  for  the  Aircraft  Computer 
Resources  Branch  (MMEC)  within  the  Engineering  Division  of  the  Directorate  of  Material  Man¬ 
agement. 

OO-ALC  has  the  System  Program  Manager  (SPM)  responsibility  for  the  F-16  fighter,  providing 
the  management  and  engineering  support  for  the  weapon  system  including  the  Avionics  Integra¬ 
tion  Support  Facility  (AISF).  The  AISF  is  an  applied  engineering  laboratory  designed  to  support 
digital  airborne  systems  and  subsystems  and  associated  programs  and  equipment. 

In  addition  to  the  Ogden  Air  Logistics  Center,  Hill  AFB  is  also  home  to  the  388th  Tactical  Fighter 
Wing,  a  Tactical  Air  Command  (TAC)  wing  flying  the  F-16A/B  aircraft  with  responsibility  for  or¬ 
ganizational  and  intermediate  level  communication,  navigation  and  avionics  systems  mainte¬ 
nance. 

OO-ALC  engineers  and  managers,  as  well  as  F-1 6A/B  maintainers  from  the  388th  Tactical  Fighter 
Wing,  contributed  to  the  discussions.  With  this  range  of  representation  from  both  software  main¬ 
tainers  and  software  users,  Hill  AFB  was  an  excellent  starting  point  for  the  fact-finding  discus¬ 
sions.  Attendees  are  snown  in  Table  2-2,  and  Appendix  B  lists  the  attendee’s  contact  infor¬ 
mation. 

The  team  was  briefed  by  Mr.  Bruce  W.  Rudd  of  the  F-16  Fire  Control/HUD  OFP  Development 
Section  (OO-ALC/MMECA)  on  post  deployment  software  support  (PDSS)  concerns  facing  Hill 
AFB  and  the  software  maintenance  community.  It  is  OO-ALC ’s  position  that  many  of  the  software 
issues  today  focus  on  the  Air  Force's  ability  to  use  and  manage  its  mission  critical  computer 
resources  (MCCR).  This  briefing  is  contained  in  Appendix  C. 

The  impact  of  maintenance  for  these  systems  is  often  given  second  place  in  the  acquisition 
arena.  OO-ALC  would  suggest  that  systems  be  designed  and  delivered  with  PDSS  in  mind. 
They  cite  three  areas  of  concern: 

•  software 

•  documentation 

•  support  tools 

In  each  of  these  areas,  the  OO-ALC  briefing  expands  on  what  can  be  done  in  these  important 
areas  to  adequately  ensure  that  a  supportable  system  is  delivered.  In  summary,  they  present 
that  the  major  problem  is  in  complying  with  and  ensuring  compliance  with  established  methods  of 
software  engineering,  and  not  necessarily  with  awareness  of  what  is  needed  to  develop  and 
deliver  supportable  software  systems. 
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Ogden  Air  Logistics  Center 

Rick  Holsman 

Aircraft  Computer  Resources 
Branch  Chief,  Engineering  Division, 
Directorate  of  Material  Manage¬ 
ment 

OO-ALC/MMEC 

Major  Gary  S.  Hochstetler 

Aircraft  Computer  Resources 
Branch  (AFRES  IMA  assignee) 

OO-ALC/MMEC 

Leon  Oldham 

F-16  Radar/Stores  OFP  Develop¬ 
ment  Section  Chief 

OO-ALC/MMECR 

Bruce  W.  Rudd 

F-16  Fire  Control/HUD  OFP  Devel¬ 
opment  Section 

OO-ALC/MMEC  A 

Robert  Sharp 

Software  Support  Center  Branch, 
Missile  and  Aircraft  Systems  Divi¬ 
sion,  Directorate  of  Maintenance 

OO-ALC/MAKTF 

Bill  Frost 

F-4  AISF  Section 

OO-ALC/MMECT 

388th  Tactical  Fighter  Wing 

TSgt.  Jeffrey  A.  Carey 

Integrated  Avionics  Branch,  388 
Component  Repair  Squadron 

388  CRS/MACIA 

TSgt.  Rudulph  W.  Peart 

Integrated  Avionics  Branch,  388 
Component  Repair  Squadron 

388  CRS/MACIA 

SSgt.  Christopher  B.  Gifford 

388  Aircraft  Generation  Squadron 

388  AGS/MAABC 

Table  2-2:  Participants  from  Hill  AFB 


2.2.  Dyess  Air  Force  Base 

The  fact-finding  team  visited  the  96th  Bombardment  Wing  (Heavy),  Dyess  AFB,  TX,  on  13  August 
1987.  The  96  BMW  is  a  Strategic  Air  Command  wing  flying  the  B-1B  bomber.  The  96  BMW  host 
for  this  meeting  was  Captain  Steve  Hackett,  Central  Integrated  Test  System  (CITS)  Maturation 
Officer  (MAMC)  within  the  Maintenance  Control  Division  reporting  to  the  Deputy  Commander  for 
Maintenance. 

Dyess  AFB  provided  additional  perspectives  into  the  nature  of  what  the  messages  for  a  software- 
oriented  BTV  might  be.  Not  only  did  the  96  BMW  provide  support  for  the  main  discussions,  they 
also  provided  thought-provoking  side  discussions  and  an  opportunity  to  tie  all  these  concepts 
together  by  visiting  the  flight  line  and  seeing  a  B-1B  bomber  in  the  hanger  undergoing  normal 
maintenance.  In  addition  to  the  wing’s  mission-oriented  emphasis  on  the  B-1B  bomber,  the  Air 
Force  Operational  Test  and  Evaluation  Center  (AFOTEC)  personnel  of  the  B-1B  FOT  &  E  Test 
Team  participated  in  these  discussions.  Attendees  are  shown  in  Table  2-3,  and  Appendix  B 
contains  the  attendee’s  contact  information.  The  main  elements  of  these  discussions  are  in¬ 
cluded  in  Section  3,  Software  Issues. 
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96th  Bombardment  Wing  (Heavy) 

Capt.  Steve  Hackett 

CITS  Maturation  Officer,  Mainte¬ 
nance  Control  Division,  Deputy 
Commander  for  Maintenance 

96  BMW/MAMC 

CMSgt.  Robert  Lewis 

NCOIC,  Maintenance  Systems 
Analysis,  Maintenance  Control  Divi¬ 
sion,  Deputy  Commander  for  Main¬ 
tenance 

96  BMW/MAP 

Air  Force  Operational  Test  &  Evaluation  Center 

Major  Gary  F.  Giesecke 

Deputy  for  Software.  AFOTEC 
B-1 B  FOT  &  E  Test  Team 

AFOTEC  FOT  &  E/TDS 

Capt.  Glenn  Tuley 

Deputy  for  Software,  AFOTEC 
B-1  B  FOT  &  E  Test  T earn 

AFOTEC  FOT  &  E/TDS 

1  Lt.  Rich  Dale 

Deputy  for  Software,  AFOTEC 
B-lB  FOT  &  E  Test  Team 

AFOTEC  FOT  &  E/TDS 

2Lt.  Emily  Andrew 

Deputy  for  Software,  AFOTEC 
B-1  B  FOT  &  E  Test  Team 

AFOTEC  FOT  &  E/TDS 

MSgt  Mel  Noble 

Maintenance  Evaluator  for  Avionics 
Maintenance  Squadron,  AFOTEC 
B-1  B  FOT  &  E  Test  Team 

AFOTEC  FOT  &  E/TDLA 

SSgt.  W.  R.  Wanamaker 

Maintenance  Evaluator  for  Avionics 
Maintenance  Squadron,  AFOTEC 
B-1B  FOT  &E  Test  Team 

AFOTEC  FOT  &  E/TDLA 

Table  2-3:  Participants  from  Dyess  AFB 


2.3.  Robins  Air  Force  Base 

The  fact-finding  team  visited  the  Warner  Robins  Air  Logistics  Center  (WR-ALC),  Robins  AFB,  GA, 
on  2-3  September  1987.  The  WR-ALC  hosts  for  these  meetings  were  Mr.  John  J.  LaVecchia, 
Branch  Chief  for  the  Engineering  and  Reliability  Branch  (MMRR)  within  the  Electronic  Warfare 
(EW)  Management  Division  of  the  Directorate  of  Material  Management,  and  Mr.  Larry  K.  Israel, 
Branch  Chief  for  the  Software  Support  Center  Branch  (MAIT)  within  the  Airborne  Electronics 
Division  of  the  Directorate  of  Maintenance. 

WR-ALC  provided  the  opportunity  to  explore  issues  regarding  types  of  weapons  systems  soft¬ 
ware  that  were  not  part  of  the  digital  avionics  software  flying  the  aircraft,  such  as  electronic 
warfare  software  and  automatic  test  equipment  (ATE)  software.  Attendees  are  shown  in 
Table  2-4,  and  Appendix  B  contains  the  attendee's  contact  information. 

By  direction  of  a  Program  Management  Directive  (PMD),  WR-ALC  establishes  the  electronic  war¬ 
fare  avionics  integration  support  facility  (EWAISF),  which  provides  services  to  achieve  organic 
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Warner  Robins  Air  Logistics  Center 

JohnJ.  LaVecchia 

Engineering  and  Reliability  Branch 
Chief,  Electronic  Warfare  (EW) 
Management  Division,  Directorate 
of  Material  Management 

WR-ALC/MMRR 

1 

Harry  Jennings 

Emergency  Reprogramming  Cen¬ 
ter,  Simulation  and  Evaluation  Sec¬ 
tion 

WR-ALC/MMRRA 

Dorothy  Jackson 

EF-111/ALQ-125  Unit,  EW  Inte¬ 
grated  Systems  Section 

WR-ALC/MMRRIC 

1 

Joseph  Watwood 

APR-38  Unit,  EW  Integrated  Sys¬ 
tems  Section 

WR-ALC/MMRRIA 

Michael  S.  Tipton 

EW  Active  Systems  Section 

WR-ALC/MMRRC 

William  Calkins 

Radar  Warning  Receiver  Section 

WR-Al  O/MMRRV 

Roy  P.  Oliver 

ALR-74/F-1 5  TEWS  Unit,  Radar 
Warning  Receiver  Section 

WR-ALC/MMRRVE 

1 

William  Raymond  Haggard 

Integration  Support  Facility  Opera¬ 
tions  Branch,  Systems  Engineering 
Division,  Directorate  of  Material 
Management 

WR-ALC/MMEC 

1 

Larry  K.  Israel 

Software  Support  Center  Branch 
Chief,  Airborne  Electronics  Divi¬ 
sion,  Directorate  of  Maintenance 

WR-ALC/MAIT 

Lonnie  Y.  Totty 

Radar  and  Space  Communications 
Section  Chief 

WR-ALC/MAITB 

1 

Jerry  L.  Watts 

Countermeasures  Unit  Chief, 
Radar  and  Space  Communications 
Section 

WR-ALC/MAITBC 

Charles  R.  Singleton 

Tactical  Support  Section  Chief 

WR-ALC/MAITC 

Jim  Mosely 

F-15  Radar  Support  Unit,  Tactical 
Support  Section 

WR-ALC/MAITCA 

Table  2-4:  Participants  from  Robins  AFB 


USAF  support  capabilities  for  all  present  and  future  reprogrammable  airborne  EW  systems.  The 
EW  Management  Division  (WR-ALC/MMR)  is  responsible  for  the  following  EW  system  elements: 

•  management  of  items  and  components 

•  hardware  engineering 

•  EW  software  engineering 

•  reprogramming  management  and  support 

•  total  EW  system  and  subsystem  engineering,  including  integration 


« 


12 


CMU/SEI-87-TR-42 


•  configuration  management 

•  documentation 

•  coordination  with  SPO  to  ensure  acquisition  of  proper  support  tools 

The  Software  Support  Center  (WR-ALC/MAIT)  performs  maintenance  and  support  for  existing 
automatic  test  equipment  (ATE)  software  for  all  items  repaired  by  WR-ALC/MAI,  and  all  items 
prime  at  WR-ALC  (items  not  repaired  by  another  ALC)  as  well  as  developing  ATE  software  and 
providing  acquisition  support  and  technology  development.  The  Software  Support  Center  is  also 
responsible  for: 

•  unit  under  test  (UUT)  test  software  support 

•  ATE  system  level  software  support 

•  technical  support  to  System  Program  Managers/Item  Managers 

•  technical  assistance  to  the  technology  repair  centers 

•  consulting  assistance  to  organizations  responsible  for  acquiring  ATE  and  support 
facilities 

•  production  of  software  and  firmware  for  distribution  to  users 

•  development  of  test  program  sets  (TPS)  for  WR-ALC  prime  systems 
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3.  Software  Issues 

This  section  attempts  to  paraphrase  the  various  concerns  and  messages  expressed  by  U.S.  Air 
Force  personnel,  both  in  and  out  of  uniform,  who  met  with  the  AFCOLR-sponsored  fact-finding 
team  at  the  various  locations.  To  cite  an  earlier  SEI  study:1 

Our  field  research  has  revealed  that  many  of  the  individuals  involved  in  software  ac¬ 
quisition  view  software  maintenance  as  substantially  similar  to  hardware  maintenance. 

This  orientation  often  fails  to  appreciate  the  complexity  of  software  maintenance,  and 
the  continuing  importance  of  acquiring  the  technology  needed  to  maintain  the  software. 

To  gain  this  kind  of  appreciation,  it  is  necessary  to  understand  the  unique  character¬ 
istics  of  software  maintenance  as  well  as  the  role  maintenance  plays  in  the  software  life 
cycle. 

The  fact-finding  team  participated  in  discussions  with  software  maintainers  and  system  users,  as 
described  in  Section  2,  to  be  able  to  understand  these  characteristics  of  software  maintenance 
and  to  be  able  to  incorporate  these  thoughts  into  the  process  of  planning  for  one  or  more 
software-oriented  Blue  Two  Visits.  The  issues  raised  by  the  participants  in  the  various  discus¬ 
sions  did  not  necessarily  describe  all  the  concerns  affecting  software  development,  maintenance, 
and  enhancement  in  Air  Force  systems  but  did  provide  insights  into  significant  technical  and 
management  considerations  of  Air  Force  software  maintainers. 

These  considerations  fell  primarily  within  six  areas.  They  are: 

1 .  weapon  system  orientation 

2.  acquisition  issues 

3.  post  deployment  software  support  issues 

4.  software  development  issues 

5.  supportability  and  testability  issues 

6.  personnel  management  issues 

Regardless  of  the  breadth  or  number  of  the  issues  raised  during  these  discussions,  the  over¬ 
whelming  attitude  of  the  Air  Force  personnel  was  that  "We  in  Maintenance  get  the  job  done  in 
spite  of 

3.1.  Weapon  System  Orientation 

One  of  the  difficult  views  to  communicate  to  defense  contractor  personnel  on  a  Blue  Two  Visit  is 
that  to  the  end  users  of  a  major  weapon  system,  it  is  just  that  —  a  system.  The  developers  or 
maintainers  may  see  it  as  a  loose  collection  of  software  and  hardware  but,  by  the  time  it  reaches 
the  field,  it  has  become  a  singular  system  designed  to  support  a  mission-critical  need.  Partic¬ 
ipants  in  these  discussions  agreed  that  there  appears  to  be  an  increasing  trend  towards  the 


'  The  Effect  of  Software  Support  Needs  on  the  Department  of  Defense  Software  Acquisition  Policy:  Part  1  -  A 
Framework  tor  Analyzing  Legal  Issues,  Anne  C.  Martin  and  Kevin  M.  Deasy,  Technical  Report  CMUVSEI-87-TR-2 
(ESD-TR-87-102),  January  1987 
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development  and  procurement  of  generic  hardware,  while  software  is  rapidly  becoming  the  distin¬ 
guishing  factor  in  today’s  weapon  systems.  Participants  reported  mat  it  currently  takes  three  to 
five  years  to  implement  a  functional  change  in  hardware,  including  time  to  get  the  modification 
kits  delivered  and  installed  on  the  aircraft.  A  software  change  can  be  implemented  on  the  aircraft 
in  twelve  to  eighteen  months  (or  less,  in  some  instances),  including  the  technical  order  changes, 
if  any.  Maintained  perceive  that  the  mindset  evident  today  among  managers  and  decision 
makers  is  that  a  software  change  is  therefore  cheaper  than  a  hardware  change.  Invariably, 
maintained  report  that  any  change  impacts  software,  and  software  updates  are  expensive. 

Software  maintenance  is  performed  at  the  depot  level,  typically  at  an  Air  Logistics  Center  (ALC) 
or  contractor  facilities.  Intermediate  shop-level  maintained  fault  isolate  boxes  down  to  the  level 
of  the  shop  replaceable  unit,  while  organizational  maintained  fault  isolate  the  systems  down  to 
the  line  replaceable  unit  (LRU).  Organizational  maintained  also  load  new  software  into  the  air¬ 
craft,  typically  by  installing  new  memory  cards  or  reloading  memory  using  a  memory 
loader/verifier.  Issues  such  as  DoD-STD-2167  and  post  deployment  software  support  are  not  of 
immediate  interest  to  the  maintained  at  the  organizational  and  intermediate  levels,  as  their  em¬ 
phasis  is  on  the  system  and  the  boxes  that  comprise  the  system.  In  fact,  these  discussions  led  to 
the  conclusion  that  there  are  several  levels  of  individuals  interested  in  some  facet  of  aircraft 
operational  and  test  software.  These  levels  include:  depot,  intermediate  and  organizational  main¬ 
tained,  and  users  preparing  mission  data  parameters  to  be  used  as  part  of  the  systems.  These 
various  levels  are  shown  in  Table  3-1 . 

Users  - 

Organizational  Use  operational  flight  programs  (OFP)  in  performance  of  unit  mission. 

Use  built-in-test  (BIT)  or  operational  test  programs  (OTP)  to  diagnose  sys¬ 
tems  to  the  level  of  a  faulty  LRU. 

Intermediate  Use  diagnostics  and  automated  test  equipment  (ATE)  to  diagnose  systems  to 
the  level  of  a  faulty  module  in  a  LRU. 

Use  ATE  systems  software  provided  with  ATE. 

Depot  Performs  intermediate  functions  in  addition  to  the  use  of  diagnostics  and  au¬ 

tomated  test  equipment  (ATE)  to  diagnose  systems  to  the  level  of  a  faulty 
component  in  a  module. 

Use  ATE  systems  software  provided  with  ATE. 

Malntalners  - 

Used  Headquarters-level  or  equivalent  preparing  mission  data  parameter  sets 

using  automated  mission  planning  systems. 

For  example,  the  Strategic  Mission  Data  Planning  System  (shown  in  Figure 
3-2)  provides  mission  data  for  the  B-1B  aircraft  OFP.  Other  similar  systems 
exist,  such  as  TAC’s  Mission  Support  System  or  the  TAWC  for  the 
AN/ALQ-131  system.  The  Area  Reprogramming  Capability  (ARC)  is  being 
developed  to  provide  this  capability  for  all  EW  systems. 

ALC  MM  offices  responsible  for  maintenance  of  operational  flight  programs  (OFP) 

and  built-in-test  (BIT). 

MA  offices  responsible  for  intermediate  and  depot  level  diagnostics  and  ATE 
systems  software. 

Table  3-1 :  Maintainer  Responsibilities  for  Software 


The  EW  systems  management  community  has  an  especially  clear  view  of  the  interrelationships 
between  these  various  roles.  They  see  the  players  in  the  flow  of  maintaining  EW  systems  as 
encompassing  all  of  the  following: 

•  Intelligence  community  -  provides  new  threat  data  from  the  operational  environment. 

•  Operational  units  -  provides  operational  requirements  based  on  mission  accomplish¬ 
ment  and  newly-recered  threat  data. 

•  Tactical  Air  Warfare  Center  (TAWC)  -  provides  operational  requirements  based  on 
mission  accomplishment  and  newly-received  threat  data. 

•  System  Program  Managers  -  provides  system  requirements  to  perform  system 
upgrades.  These  requirements  are  reported  to  come  from  users  in  the  field,  gener¬ 
ated  from  intelligence  data  by  the  TAWC  or  generated  by  the  ALC  itself. 

•  Air  Logistics  Center  -  performs  engineering  efforts  necessary  to  make  modification, 
test  modification,  and  document  and  use  depot  resources  to  prepare  for  shipment  to 
the  field,  as  shown  in  Figure  3-1. 

•  Organizational  and  Intermediate  maintainers  -  install  modifications  into  systems  and 
maintain  systems. 


Figure  3-1 :  ALC  Efforts  in  AISF  and  Software  Support  Centers 


For  these  reasons,  the  issues  identified  by  participants  in  these  discussions  and  listed  below  tend 
to  address  not  only  software,  but  also  the  processes  of  acquiring,  developing  and  maintaining 
software-intensive  systems.  These  issues  also  have  impacts  on  other  areas.  For  example,  in  the 
area  of  documentation,  the  decision  to  reduce  the  numbers  of  deliverable  documents  and  the 
expected  contents  of  these  documents  as  means  of  managing  cost  and  schedule  (without  ade¬ 
quately  controlling  the  underlying  process)  may  have  impacts  evidenced  by  pooler  software  de- 
•  velopment  quality,  greater  post  deployment  software  support  costs  and  greater  difficulties  in  ade¬ 

quately  testing  the  system.  This  is  why  it  is  so  important  that  the  acquisition  personnel  who  are 
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overseeing  systems  development  and  delivery  also  need  to  be  educated  as  to  software’s  life 
cycle  needs.  No  matter  how  well  educated  the  contractors  are,  it  m^y  not  make  enough  dif¬ 
ference  unless  program  managers  make  Pr>SS  concerns  a  priority  in  the  acquisition. 


3.2.  Acquisition  Issues 

Certain  individuals  participating  in  the  fact-finding  discussions  felt  that  more  acquisition  problems 
were  at  the  managerial  level  than  at  the  technical  level.  They  felt  that  educating  acquisition 
personnel  only  solves  part  of  the  problem.  They  suggested  that  although  the  DoD  has  the  most 
technologically  complex  systems,  there  are  not  enough  credible,  technically  competent  people  in 
the  government  working  in  the  acquisition  arena.  The  example  cited  here  was  that  technical 
oversight  is  often  tasked  too  far  down  within  the  acquisition  organizations,  primarily  because 
there  are  too  few  people  in  the  acquisition  offices  and  too  great  a  workload  for  these  people  to 
handle. 

Participants  at  all  locations  felt  that  acquisition  personnel  need  to  do  a  better  job  of  specifying  the 
software  systems,  tailoring  the  procurement  (including  requirements  and  reference  documents), 
and  then  enforcing  compliance  with  the  requirements  as  written.  They  felt  that  it  is  necessary  for 
acquisition  personnel  to  understand  the  processes  that  they  are  responsible  for  managing.  This 
was  pointed  cut  several  times  during  discussions  that  too  often  cost  and  schedule  needs  drive 
changes  which  are  made  by  cutting  documents  or  contents,  rather  than  better  controlling  the 
development  process.  One  point  made  was  that  documents  should  not  be  thought  of  as  separate 
products,  but  as  an  integral  part  of  the  engineering  efforts.  Another  viewpoint  expressed  during 
these  discussions  is  that,  although  acquisition  personnel  are  driven  by  cost,  schedule  and  perfor¬ 
mance  issues,  they  need  to  be  convinced  that  the  standardized  formats  specified  by  the  Data 
Item  Descriptions  (DID)  should  be  adhered  to.  Eventually,  all  the  documentation  produced  by  the 
prime  contractors  and  the  sub-tier  contractors  gets  turned  over  to  the  ALCs  for  support  of  the 
systems.  Depot-level  maintainers  reported  that  advances  like  MIL-STD-1840,  Initial  Graphic  Ex¬ 
change  Specification  (IGES),  Standard  Generalized  Markup  Language  (SGML)  and  other  efforts, 
such  as  Computer-Aided  Acquisition  and  Logistics  Support  (CALS),  will  indeed  b  seful.  Depot 
personnel  believe  the  weapon  system  software  development  process  represented  by  DoD- 
STD-21 67  to  be  a  comprehensive  and  robust  process  when  the  design  and  documentation  are 
built  first  and  detaiiea  design  and  code  follows. 

It  was  felt  that  the  acquisition  community  does  not  have  the  best  tools  available  to  support  ac¬ 
quisition  efforts  in  both  the  planning  and  documentation  aspects.  The  acquisition  community 
needs  to  have  an  available  library  that  will  support  building  a  cohesive  and  comprehensive  con¬ 
tract,  statement  of  work,  and  functional  requirements.  The  problem  is  identifying  what  the  system 
and  the  software  must  do  and  then  ensuring  that  the  proper  standards,  specifications  and 
deliverables  are  included  in  the  contract.  This  may  be  a  future  action  for  Headquarters,  Air  Force 
Systems  Command,  to  take  the  lead  in  establishing  such  an  entity,  with  inputs  coming  from  all 
levels  of  system  responsibility  (i.e.,  using  commands,  ALCs,  AFALC,  SPOs  and  contractors). 
The  INQUIRE  system  being  developed  under  the  Policy  and  Procedure  Guidance  Project  of  the 
Computer  Resource  Management  Technology  Program  (PE64740F)  is  a  start  towards  these 
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kinds  of  tools.  These  tools  could  augment  the  abilities  of  the  acquisition  personnel.  The  feeling 
of  some  participants  was  that  these  tools  are  especially  needed  since  there  are  too  few  tech¬ 
nically  literate  individuals  in  the  acquisition  process.  For  example,  in  the  B-1B  procurements,  the 
omission  of  one  word  in  a  particular  contract  (as  compared  to  another  contract  for  another  system 
which  had  to  function  with  this  system)  is  reported  to  have  changed  the  test  philosophy  for  one  of 
the  major  contractors.  Part  of  the  aircraft  systems  have  been  designed  to  "detect  and  isolate 
failures  on  the  aircraft  while  in  flight",  while  other  on-board  systems  were  designed  to  "detect 
failure  in  flight  and  fault  isolate  on  the  ground." 

Not  only  do  acquisition  personnel  need  to  understand  acquisition  issues,  participants  commented 
that  acquisition  personnel  must  personally  understand  the  processes  that  they  are  responsible  for 
procuring,  i.e.,  the  operational  aspects  of  the  systems,  instead  of  relying  solely  upon  inputs  from 
the  user.  The  Ogden  ALC  reports  substantial  benefits  have  been  achieved  through  the  use  of 
operational  personnel  from  the  co-located  TAC  wing  in  cooperation  with  the  developers.  These 
efforts  have  used  the  test  stations  and  integration  facilities  to  coordinate  and  evaluate  proposed 
changes  to  the  pilot-machine  interface.  Participants  strongly  felt  that  there  needs  to  be  a  more 
effective  use  of  DO  personnel  in  development  efforts  as  part  of  a  co-located  team,  rather  than 
having  to  fly  operators  and  pilots  in  from  operating  locations.  The  importance  of  early  involve¬ 
ment  by  operational  personnel  was  repeatedly  stressed  since  it  is  too  late  to  make  a  cost- 
effective  impact  on  system  development  by  the  time  the  acquisition  has  reached  its  Preliminary 
Design  Review  or  Critical  Design  Review. 

Acquisition  personnel  (Program  Managers  and  Deputy  Program  Managers  for  Logistics  (DPML)) 
need  to  get  the  ALC/MM  involved  earlier  in  the  program  life  cycle.  Typically,  the  DPMLs  are 
overworked,  and  the  ALCs  have  the  insight  into  and  the  experience  with  PDSS  issues.  The 
suggestion  was  made  that  all  DPMLs  work  at  an  ALC  for  a  three  year  training  assignment  before 
moving  into  their  first  DPML  assignment.  The  96th  BMW  reports  that  they  have  successfully 
integrated  the  Oklahoma  City  ALC  into  certain  of  their  working  group  meetings. 

It  is  felt,  however,  that  program  managers  are  typically  not  rewarded  for  really  caring  about  the 
quality  of  maintenance.  Within  the  current  DoD  acquisition  and  personnel  management 
schemas,  there  is  no  way  to  reward  the  program  managers  for  the  quality  of  software  mainte¬ 
nance  (or  any  other  maintenance)  of  fielded  systems.  Their  greatest  concern  is  to  get  the  system 
built  on  time  and  within  budget.  The  major  drivers,  in  order  of  importance,  often  appear  to  be: 
cost/schedule,  performance,  usability,  then  supportability.  Program  managers  are  primarily 
rewarded  only  for  meeting  schedule  and  budget  requirements  of  his  project;  thus,  the  support 
aspect  in  general  is  the  first  to  be  cut  or  ignored.  In  addition,  it  was  expressed  that  the  inclusion 
of  maintenance  aids  is  often  viewed  by  upper  levels  of  management  as  "gold  plating."  However, 
the  opposite  is  true.  While  these  aids  may  cost  more  in  the  short  term  (in  the  system’s 
acquisition),  they  can  ultimately  reduce  system  life  cycle  costs  in  the  long  term. 

Several  participants  in  these  discussion  felt  that  there  needs  to  be  a  single  responsible  focal  point 
for  software  during  the  development  of  a  system.  It  was  suggested  that  each  program  office  have 
a  designated  Deputy  Program  Manager  for  Software  (DPMS)  who  would  be  responsible  for  the 
software  portion  of  the  acquisition  program,  much  as  the  DPML  is  responsible  for  the  logistics 
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portion.  A  responsibility  of  the  DPMS  would  be  to  increase  the  emphasis  placed  on  software 
evaluation  of  characteristics  such  as  software  testability  and  software  maintainability.  They  felt 
that  the  functionality  of  software  is  currently  addressed  by  other  test  efforts,  but  the  evaluation  of 
the  software  and  its  supporting  documentation  to  meet  PDSS  needs  is  not  currently  given  as 
much  emphasis.  They  was  suggested  that  these  efforts  be  incorporated  as  part  of  the  formal  test 
process  under  the  cognizance  of  the  DPMS.  Regardless  of  how  the  idea  of  a  DPMS  is  viewed,  it 
was  agreed  that  the  Air  Force  must  adopt  a  systems  approach  to  acquisition,  but  with  a  software 
emphasis. 

In  addition,  an  infrastructure  consisting  of  training,  courses,  schools,  and  distinct  career  paths  for 
software  acquisition  personnel  does  not  exist.  The  data  processing  (or  information  management) 
training  provided  to  an  Information  Systems  (AFSC  49XX)  officer  is  often  felt  to  be  inadequate  as 
preparation  for  dealing  with  hard  real-time  embedded  computer  systems.  Several  participants  felt 
that  we  have  too  few  technically  literate  people  in  the  acquisition  process.  The  need  is  not 
necessarily  for  experts,  but  for  individuals  who  are  technically  literate  in  the  entire  software  life 
cycle,  including  PDSS. 

Acquisition  of  the  systems  needs  to  be  better  coordinated  according  to  participants.  In  the  case 
of  the  ALR-69  radar  warning  system  for  the  F-4  and  F-16  fighters  described  to  the  fact-finding 
team,  the  hardware  boxes  were  reported  to  have  arrived  at  the  squadrons  before  the  test  stands 
and  technical  orders.  Once  systems  such  as  these  arrive  at  the  squadrons,  technical  order 
changes  arrive  through  the  standard  technical  order  distribution  officer  (TODO)  channels,  but 
software  changes  arrive  through  other  maintenance  channels.  Thus,  there  is  often  an  incompat¬ 
ibility  between  the  fielded  software  and  the  documentation.  In  another  instance,  TAC  is  reported 
to  have  asked  the  ALC  to  delay  releasing  a  block  change  to  an  aircraft  OFP  because  of  the 
impact  caused  by  retraining  for  the  modified  pilot-machine  interface.  The  B-1B  maintenance 
training  sets  (a  Systems  Maintenance  Training  Set  and  a  Avionics/Armament  Maintenance  Train¬ 
ing  Set)  are  reportedly  not  consistent  with  the  configuration  baseline  of  the  aircraft.  The  training 
sets  are  at  Baseline  2,  while  the  aircraft  has  moved  past  Baseline  30.  Participants  did  not  at¬ 
tempt  to  assign  fault  in  these  areas,  either  to  operational  commands,  program  offices,  or  to  con¬ 
tractors.  Instead,  they  felt  that  the  problems  were  indicative  of  a  general  tendency  to  look  first  at 
hardware,  then  software  and  finally,  at  support  for  the  systems.  Participants  suggested  that  there 
needs  to  be  a  single  installation  date  for  any  new  system,  when  the  system,  supporting  equip¬ 
ment  and  documentation,  test  equipment,  software  models,  software  and  hardware  maintenance 
capabilities  and  the  mission  data  generation  capabilities  are  all  available  at  one  time  for  all  the 
parties  involved.  There  was  also  an  awareness  that  perhaps  this  procurement  model  would 
unnecessarily  delay  fielding  of  mission-critical  systems  while  they  waited  for  support  systems  to 
catch  up. 

A  related  issue  is  that  the  operating  commands  typically  have  net  established  a  central  program 
management  office.  For  example,  the  Deputy  Commander  for  Operations  (DO)  is  responsible  for 
simulators,  but  the  Deputy  for  Logistics  (LG)  is  responsible  for  test  sets.  A  central  office  would 
provide  a  focus  for  people  to  take  a  weapon  system  perspective  and  manage  the  evolution  of  the 
system  in  a  top-down,  integrated  fashion.  Early  establishment  of  such  a  program  management 
office  would  allow  using  command  participation  earlier  in  the  development  efforts,  as  suggested 
above. 
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3.3.  Post  Deployment  Software  Support  Issues 

One  participant  estimated  that  sixty  to  seventy  percent  of  the  life  cycle  costs  for  a  software¬ 
intensive  system  are  found  in  post  deployment  software  support  (PDSS).  PDSS  is  the  sum  of  all 
activities  required  to  ensure  that,  during  the  production/deployment  phase  of  a  mission-critical 
computer  system’s  life,  the  implementation  and  fielded  software/system  continues  to  support  its 
original  operational  mission  and  subsequent  mission  modifications  and  product  improvement  ef¬ 
forts.  Several  participants  in  our  discussions  felt  that  altogether  too  often,  when  life  cycle  costs 
are  addressed,  the  majority  of  the  attention  is  being  focused  on  up-front  hardware  costs,  effec¬ 
tively  delaying  software  costs  into  the  PDSS.  During  the  discussions  at  Hill  AFB,  Mr.  Bruce  Rudd 
briefed  the  OO-ALC/MMEC  position  on  PDSS  issues.  This  briefing  is  found  in  Appendix  C. 

One  concern  raised  by  participants  deals  with  the  contractor's  perspective  towards  PDSS.  Cur¬ 
rently,  contractors  willingly  sell  manhours  to  support  fielded  systems  that  they  developed  or 
helped  to  develop.  In  fact,  it  was  felt  that  contractors  may,  knowingly  or  unknowingly,  allow  this  to 
affect  the  quality  of  their  efforts  during  system  development.  However,  some  maintainers  re¬ 
ported  concerns  that  in  today’s  business  world  with  shortages  of  key,  talented  personnel,  contrac¬ 
tors  may  be  forced  or  choose  to  put  all  or  most  of  their  efforts  into  major  new  programs,  like  the 
Advanced  Tactical  Fighter,  effectively  reducing  the  supportability  of  current  weapon  systems 
which  have  depended  un  procuring  continuing  contractor  support. 

Documentation  is  considered  to  be  a  common  problem.  Depots  report  receiving  documentation 
that  has  been  written  directly  from  the  completed  code.  The  concern  here  is  that  it  is  difficult  to 
get  software  designers  to  document  their  efforts.  This  is  not  necessarily  true  because  of  some 
personality  failure  of  software  designers,  but  that  the  tools  to  easily  allow  capturing  the  design 
data  and  design  decisions  are  not  readily  available  in  the  domain  in  which  they  normally  work.  It 
was  felt  that  a  hardware  design  lends  itself  to  being  documented  on  paper,  but  that  a  software 
design  needs  to  have  a  way  of  being  captured  electronically  while  the  software  engineer  works  at 
the  terminal.  Participants  agreed  that  this  can  be  a  major  cost  driver;  however,  sound  documen¬ 
tation  principles  should  not  be  minimized.  In-house  ALC  software  efforts  stress  documentation  at 
all  levels  of  the  development.  The  use  of  Ada®  was  highly  stressed  as  an  aid  to  the  documen¬ 
tation  problems.  This  does  not  imply  that  Ada  itself  is  self  documenting,  but  the  use  of  the 
language  lends  itself  to  documentation  "as-you-go." 

Deliverable  documentation  must  be  traceable  and  useful  for  PDSS  personnel.  Different  number¬ 
ing  schemes  for  requirements,  design,  code  and  test  cases  and  data  invariably  lead  to  confusion 
and  additional  effort  to  correctly  cross-reference  these  items.  The  software  maintainer’s  greatest 
difficulties  are  in  finding  "where  something  exists."  This  may  be  anything  from  a  simple 
parameter  to  a  specific  embedded  function  or  a  function  performed  in  the  code  or  documentation. 
An  additional  problem  cited  is  the  emergence  of  flawed  "boilerplate"  text.  This  text  may  be  incor¬ 
rect  or  it  may  be  a  meaningless  standard  prologue  (or  "blurb”)  that  gets  copied  everywhere  with 
only  the  name  of  the  module  changing.  Standardization  would  be  less  of  a  problem  if  methods 
such  as  IGES  or  SGML  were  employed  during  software  development  phases  of  the  system  life 
cycle. 
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3.4.  Software  Development  Issues 

A  consistent  software  design  methodology  needs  to  be  applied  throughout  the  system  devel¬ 
opment  for  all  related  components  of  a  system.  For  the  B-1B  Central  Integrated  Test  System 
(CITS)  (shown  in  Figure  3-2),  a  matrix  was  attempted  that  would  show  the  compatibility  of  the 
various  software-intensive  systems  based  on  these  attributes: 

•  programming  language 

•  coding  standards 

•  design  philosophy/methodology 

•  CITS  philosophy 

•  CITS  tools 

In  many  cases,  the  compatibility  of  these  programs  or  versions  of  programs  was  found  to  be  less 
than  complete  or  unknown. 

Generally  accepted  software  engineering  principles  need  to  be  applied  to  the  development  of  line 
replaceable  units  (LRU)  —  the  black  boxes.  Often  the  software  that  is  hidden  inside  these  boxes 
is  not  developed  by  a  software-oriented  engineer,  but  by  electrical  engineers,  mechanical  engi¬ 
neers  or  aeronautical  engineers:  all  of  whom  tend  to  be  hardware-oriented.  These  boxes  (and 
their  associated  software)  have  to  be  maintained  over  the  entire  life  time  of  the  weapon  system; 
thus,  it  was  suggested  that  more  influence  by  software  specialists  would  greatly  improve  the 
quality  of  software. 

Participants  suggested  that  greater  attention  on  the  issues  of  software  usability  and  software 
maintainability  needs  to  occur  during  software  development.  Issues  such  as  operability,  "user 
friendliness"  and  response  time  need  to  be  substantively  addressed  as  the  system  is  developing. 
Also  needing  consideration  is  the  area  of  software  security  and  the  operations  impact  that  secure 
computing  has.  For  example,  the  B-1 B  has  eight  primary  computers  and  its  data  cartridges  must 
be  removed  and  replaced  by  a  maintainer.  The  Strategic  Mission  Data  Planning  System 
(SMDPS)  data  cartridges  provided  to  the  B-1  B  aircraft  (as  shown  in  Figure  3-2)  can  contain  highly 
classified  data. 

Mission  data  must  also  be  maintained  over  the  lifetime  of  the  system.  Participants  felt  that  this 
area  was  often  not  well  understood  or  addressed  in  the  design  of  complex  systems.  It  was  felt 
that  operational  software  must  be  designed  with  a  clear  separation  between  the  algorithmically- 
oriented  operational  flight  programs  (OFP)  and  the  mission  data,  as  shown  in  Figure  3-3.  The 
mission  data  tends  to  be  more  volatile  than  the  OFP,  and  is  not  necessarily  "just”  a  few 
parameters  that  can  easily  be  hardcoded  into  the  software.  In  the  EW  Avionics  Integrated  Sup¬ 
port  Facility  (EWAISF),  the  team  members  were  shown  a  set  of  mission  data  parameters  for  a 
particular  radar  warning  receiver.  This  collection  of  controlled,  classified  data  was  best  measured 
in  terms  of  inches  of  hardcopy,  rather  than  as  numbers  of  parameters.  The  magnitude  of  these 
kinds  of  changes  are  best  understood  in  light  of  an  example  of  a  particular  EW  system  described 
at  WR-ALC.  A  particular  change  in  the  unit  required  replacing  soldered  PROMs  on  over  20 
printed  circuit  boards  in  approximately  2000  fielded  units.  In  addition  to  the  logistical  issues  in 
distributing  these  changes,  there  are  also  reliability  and  maintainability  issues  to  be  considered. 
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Figure  3-3:  Desired  Separation  of  OFP  and  Mission  Data 

In  addition,  a  common  Air  Force  software  integration  laboratory  or  environment  is  not  currently 
available.  The  B-1B  Program  Office  was  forced  into  assuming  the  role  of  system  integrator,  due 
to  funding  constraints  which  prevented  paying  a  contractor  to  perform  this  role.  Due  to  this,  the 
government  has  had  to  accept  the  products  of  numerous  development  environments  and  then 
integrate  the  weapon  system.  In  the  B-1B  environment,  this  has  led  to  the  establishment  of  the 
CITS  Maturation  team.  This  group  of  six  officers  and  senior  NCOs  is  functioning  in  an  integration 
and  trouble-shooting  role.  The  original  planned  approach  for  CITS  maintenance  data  processing 
is  already  showing  signs  of  change.  The  CITS  Expert  Parameter  System  (CEPS),  an  Al-based 
expert  system  currently  being  developed,  and  the  Filter  Program  are  additions  to  the  overall 
system  shown  in  Figure  3-2. 

Also  impacting  the  B-1B  maintenance  efforts  is  the  Core  Automated  Maintenance  System 
(CAMS),  shown  in  Figure  3-2.  CAMS  is  being  developed  by  the  Air  Force  Standard  Systems 
Center,  Gunter  AFS,  under  a  Program  Management  Directive  from  HQ  USAF/LEY.  It  is  a  com¬ 
mon  Air  Force-wide  system  replacing  comparable  systems  that  used  to  exist  at  the  major  com¬ 
mands.  Its  implementation  is  being  driven  by  the  planned  CAMS  schedule.  Previous  releases  of 
CAMS  have  had  failings.  Fixes  are  being  promised  in  future  releases  and  are  not  being  released 
separately,  but  are  being  incorporated  into  the  next  scheduled  release.  The  next  release  has 
currently  had  four  scheduled  delivery  dates.  Problems  with  this  system  have  been  receiving 
senior  management  attention.2  HQ  SAC  requested  that  the  CAMS  baseline  be  frozen  and 
solidified  (i.e.,  made  to  work)  before  proceeding  with  enhancements.  User’s  suggested  enhance¬ 
ments  or  fixes  appear  to  be  at  least  a  year  away  from  reaching  the  field,  while  errors  in  the  CAMS 


Message  031834Z  Aug  87,  HQ  SAC  to  SSC  Gunter/CC  (Commanding  Officer)  with  copy  to  SSC  Gunter/AQM, 
Subject:  Accelerated  Fix  of  Critical  CAMS  Problems 
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system  are  causing  the  loss  of  maintenance  records  totaling  five  to  six  thousand  man  hours  per 
month.  The  96th  BMW  has  reportedly  lost  records  for  over  31 ,000  man  hours  of  B-1 B  mainte¬ 
nance  actions.  The  fix  to  resolve  this  is  reported  to  be  a  one  line  code  change,  but  it  has  not  yet 
been  distributed,  as  the  next  release  is  not  ready  for  distribution. 

The  current  testing  practices,  including  independent  validation  and  verification  (IV&V)  based  on 
sampling,  are  reported  by  participants  as  leaving  much  to  be  desired  in  today’s  integrated  envi¬ 
ronment.  All-up  end-to-end  system  testing,  though  expensive,  is  necessary  to  insure  operability 
of  the  complete  weapon  system.  It  was  suggested  that  Program  Managers  need  to  understand 
that  IV&V  funds  are  not  a  negotiable  item.  These  efforts  are  important,  but  costly. 

3.5.  Supportability  and  Testability  Issues 

It  was  commonly  reported  that  system  design  to  support  testability  is  often  lacking,  in  the  view  of 
organizational  maintained.  Built  in  test  (BIT)  often  fault  isolates  at  too  high  a  level  —  identifying 
multiple  LRUs  as  faulty  —  effectively  forcing  the  technician  to  "shotgun"  replace  an  entire  am¬ 
biguity  group  in  order  to  return  the  aircraft  to  a  ready  state.  This  situation  is  especially  true  in 
multiple  failure  situations,  where  the  probable  cause  of  failure  identified  by  the  test  station  or  BIT 
is  not  adequate.  Often  in  these  cases,  there  are  also  not  enough  test  points  to  be  able  to  meet 
the  acquisition  requirements  regarding  fault  isolation  to  a  single  replaceable  unit  within  a  given 
probability. 

It  was  felt  that  Integrated  Diagnostics  is  not  getting  the  attention  up  front  in  the  systems  life  cycle 
that  it  should.  It  was  acknowledged  that  functional  requirements  (i.e,  fly  the  aircraft)  get  the 
attention  during  acquisition.  On  the  other  hand,  BIT  doesn’t  fly  the  airplane  and  designing  for 
testability  generally  adds  to  the  development  costs. 

The  Air  Force  is  pursuing  currently  a  five  year  development  effort  with  ten  contractors  involved  in 
this  area  to  develop  a  Generic  Integrated  Maintenance  Diagnostics  System  (GIMADS).  Partic¬ 
ipants  felt  that  a  common  strategy  or  a  standard  for  maintenance  software  is  needed,  but  that  the 
pressure  to  use  off-the-shelf  components  (as  in  the  mandate  to  maximize  use  of  F-1 6  compo¬ 
nents  in  the  B-1B)  could  limit  the  effectiveness  of  such  an  approach. 

A  problem  which  has  been  magnified  by  software,  rather  than  reduced,  is  the  "Could  Not 
Duplicate"  (CND)  class  of  failures.  These  reportedly  account  for  approximately  twenty  to  thirty 
percent  of  all  failures  received  in  the  shops.  In  order  to  keep  adequate  spares  on  hand  to  keep 
the  aircraft  flying  while  CND  boxes  are  checked  out  unnecessarily,  additional  inventory  is  brought 
into  the  logistics  channels.  For  one  system  alone,  this  accounted  for  over  a  million  dollars  in 
spares.  Another  system  was  reported  to  have  required  an  expenditure  of  7.42  million  dollars  for  a 
set  of  eleven  black  boxes.  The  maintainer’s  ideal  state  is  to  have  a  system  that  delivers  a  high 
mean  time  between  failures  and  a  low  CND  count.  A  program  undertaken  at  Hill  AFB  to  address 
this  problem  is  the  "Bad  Actor"  program.  LRUs  which  have  failed  twice  on  the  aircraft,  but  have 
passed  tests  at  the  intermediate  shop,  are  sent  directly  to  the  prime  ALC’s  Engineering  Division. 
This  bypasses  the  depot  level  intermediate  shop,  which  uses  the  same  test  sets  and  test  software 
as  the  intermediate  shop  at  the  wing.  While  this  process  flow  is  valid,  efforts  should  insure  that 
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depot  level  maintenance  personnel  be  directly  involved  in  the  programs  that  they  have  the 
greatest  combination  of  historical  and  experience  base  in  identifying  problem  areas. 

From  comments  made  by  the  participants,  it  appears  that  the  SPOs  need  a  better  understanding 
of  the  need  for  competent  technical  oversight,  and  that  they  should  plan  for  at  least  ten  percent  of 
the  acquisition  resources  dedicated  to  IV&V.  The  concern  here  is  twofold.  First,  there  often  is  a 
perception  that  there  is  not  enough  technical  expertise  in  the  SPOs  to  know  if  the  support  being 
acquired  is  sufficient  and  useful  to  the  maintenance  community.  Secondly,  there  are  concerns 
about  the  development  and  retention  of  some  "corporate  knowledge"  about  the  system.  It  was 
felt  that  individual  engineers  develop  competencies  about  the  system  under  development,  but 
that  this  knowledge  may  not  necessarily  remain  in  the  program  offices  as  personnel  transfer  out. 
For  these  reasons,  it  was  suggested  that  the  acquiring  command  might  "contract"  this  effort  out, 
either  within  the  service  to  an  ALC  or  outside  the  sen/ice  to  an  appropriate  contractor  —  either  a 
contractor  or  a  federally-funded  research  and  development  center. 

3.6.  Personnel  Management  Issues 

Personnel  management  and  assignment/rotation  issues  surfaced  in  several  of  the  discussions. 
Maintainors  report  that  from  their  perspectives  the  acquisition  personnel  repeatedly  make  the 
same  mistakes,  but  also  that  every  contact  with  the  SPO  is  always  with  new  people.  Other 
concerns  mentioned  dealt  with  the  lack  of  experienced  people  in  acquisition  offices,  especially 
experienced  logisticians.  As  mentioned  in  Section  3.2,  it  was  suggested  that  all  DPMLs  work  at 
an  ALC  for  a  three  year  training  assignment  before  moving  into  their  first  DPML  assignment  with 
Air  Force  Acquisition  Logistics  Center  (AFALC). 

Personnel  concerns  were  heard  repeatedly  in  the  discussions  at  the  ALCs.  Most  of  the  organi¬ 
zations  had  a  considerable  concentration  of  hardware  engineers,  mostly  electrical  engineers,  with 
some  other  engineering  disciplines  represented.  In  general,  there  were  few  computer  scientists 
represented  within  the  engineering  population.  This  was  reportedly  attributable  to  the  civilian 
personnel  management  practices  that  required  all  engineers  (even  those  filling  software 
engineering-related  positions)  to  have  completed  a  certain  number  of  course  units  in  the  tradi¬ 
tional  engineering  disciplines  —  courses  that  may  not  be  part  of  a  typical  computer  science 
curriculum. 

The  impact  of  these  personnel  management  policies  is  felt  within  the  AFLC  as  it  adapts  to  meet 
increasing  requirements  to  maintain  and  modify  software.  The  EW  Management  Division  at 
Warner  Robins  showed  the  team  electronic  warfare  pods  that  required  changes  and  modifications 
by: 

•  Removing  and  replacing  hardware  circuit  cards  in  a  shop 

•  Adjusting  potentiometer  settings  on  circuit  cards  in  a  shop 

•  Removing  and  replacing  memory  cards  in  a  shop 

•  Reprogramming  memory  while  the  pod  is  mounted  on  the  aircraft  using  a  memory 
loader/verifier 


26 


CMU/SE1-87-TR-42 


These  EW  pods  have  progressed  from  being  a  strictly  hardware  device  to  units  containing  ten  or 
more  microprocessors,  each  with  its  own  OFP.  As  these  systems  evolve  with  increasing  func¬ 
tionality  through  increased  software  capability,  a  corresponding  increase  in  software  capability  is 
required  within  the  personnel  of  the  Air  Logistics  Centers.  This  capability  was  characterized  by 
one  manager  as  being  necessarily  organic  within  the  ALC,  and  as  requiring  higher  skill  levels. 

While  needing  more  highly  skilled  individuals,  managers  report  that  they  have  experienced  high 
attrition  among  the  journeyman-level  engineers.  These  rates  were  reported  to  have  been  as  high 
as  18  per  cent  attrition  annually  among  GS-12  engineers.  Approximately  half  of  those  leaving 
reportedly  left  as  a  result  of  lack  of  professional  or  monetary  progression,  or  due  to  salary  im¬ 
balances  between  government  and  industry.  The  GS-12  level  engineer  is  reportedly  stifled  by  a 
lack  of  progression  since  the  engineer  must  either  stay  at  GS-1 2  or  move  into  supervisory  and 
management  positions. 

Another  issue  raised  was  the  perceived  low  self  esteem  of  many  government  employees.  This  is 
best  evidenced  by  the  old  saying,  "It's  good  enough  for  government  work."  People  don’t  neces¬ 
sarily  want  to  do  work  that  is  just  "good  enough,"  but  that  may  be  all  that  they  are  able  to  do  with 
the  available  resources.  One  organization  visited  was  authorized  over  ten  per  cent  more  person¬ 
nel  than  they  had  been  appropriated  budget  for,  and,  in  addition,  currently  had  almost  another  ten 
per  cent  of  their  positions  vacant  awaiting  hiring  actions.  This  same  organization  is  responsible 
for  maintaining  one  hundred  and  seventy-five  systems  with  over  4500  software  programs.  Of 
these  programs,  about  320  are  modified  each  year  and  approximately  one  hundred  and  fifty  new 
programs  are  written  each  year.  Not  only  are  there  over  4500  software  programs  that  this  organi¬ 
zation  is  responsible  for,  but  these  programs  account  for  58  different  languages  and  systems 
software  packages  on  over  50  different  computers.  This  proliferation  of  programs,  languages, 
and  computer  systems  dictates  the  need  for  a  large,  technically  competent  work  force. 

The  real  problem,  as  perceived  by  some  managers,  is  "inflexibility."  They  feel  the  need  to  have 
flexibility  in  getting  the  people  and  the  resources  that  they  need,  based  on  some  economic  justifi¬ 
cation,  much  more  like  a  business  is  managed.  The  perceived  reality  of  managing  within  the 
acquisition  and  maintenance  communities  is  not,  "Who  can  I  best  get  to  do  this  job?"  but,  "What 
did  l  get?".  Managers  report  that  they  need  to  be  made  accountable  for  their  efforts,  and  then 
given  the  tools  and  responsibility  to  see  that  the  job  is  competently  performed  within  cost  and 
schedule  constraints.  The  desire  to  do  the  job,  and  to  do  it  properly,  is  evident,  but  the  resources 
to  do  this  may  not  be  available  within  an  environment  which  limits  the  number  of  people  who  may 
be  hired. 
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4.  Recommendations 


4.1.  Software  Blue  Two  Visit  Messages 

Historically,  software  is  one  area  that  has  not  been  as  well  understood  during  weapon  system 
development  as,  perhaps,  it  should  have  been.  Whereas  hardware  aspects  are  now  being 
redefined  and  understood,  software  programming  functions  have  always  been  viewed  as  an  "in 
the  closet"  operation  —  where  the  programmer  was  often  left  alone  to  magically  develop  a  logical 
structure  which  eventually  went  to  satisfy  the  requirement.  Thus  in  most  of  the  earlier  weapon 
system  programs,  little  or  no  emphasis  was  given  to  the  functionality,  supportability  and  con¬ 
sequently.  the  usefulness  of  the  software  until  the  system  was  turned  over  to  the  using  command. 
Today  there  is  an  increasing  effort  within  DoD  to  correct  these  past  indifferences  and  turn  atten¬ 
tion  not  only  to  hardware  reliability  and  maintainability  but  also  to  the  critical  aspects  of  software. 

However,  to  effectively  change  present  attitudes,  the  approach  must  go  beyond  improving  and 
streamlining  of  the  acquisition  process  —  it  must  be  a  conscientious  effort  on  the  part  of  govern¬ 
ment  and  industry  to  scrutinize  the  environment  in  which  software  is  developed  and  maintained 
so  that  critical  support  issues  are  identified  and  dealt  with. 

This  section  summarizes  these  critical  messages  from  Air  Force  maintainers  to  industry’s  soft¬ 
ware  development  community  that  the  AFCOLR  Software  BTV  fact-finding  Team  heard  in  their 
visits  to  the  various  Air  Force  units. 

Suggested  areas  of  focus  for  a  software-oriented  BTV  included: 

•  operational  flight  program  software 

•  avionics  operational  flight  program  software 

•  built  in  test  within  the  overall  scope  of  operational  flight  program  software 

•  SAC,  TAC  and  ALC  mission  areas 

•  EW  operational  flight  program  software 

•  PDSS  maintainers 

•  software  users  (operational  issues) 

•  software  maintainers  (maintenance  and  PDSS  issues) 

•  automatic  test  equipment  software 

•  Integrated  Diagnostics 

•  C3!  software 

•  software  support  environments  (SSE) 

Within  these  focus  areas  there  are  relevant  issues  that  relate  to: 

•  system  orientation 

•  acquisition/support  policies  and  practices 
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•  post  deployment  software  support 

•  Documentation 

•  software  development 

•  integration 

•  interoperability 

•  compatibility 

•  supportability  and  testability 

•  maintainability 

•  commonality 

•  between  weapon  systems  (aircraft) 

•  between  sub-systems  (RADAR,  EW,  communications,  etc.) 

•  between  ATE  modules  (intermediate  and  depot  level  congruence) 

•  personnel  management 

Not  only  did  the  fact-finding  team  hear  messages  that  could  be  conveyed  to  contractors,  but  the 
team  also  heard  suggestions  that  an  Air  Force  “Tiger  Team'*  be  staffed  and  empowered  to 
resolve  issues  in  these  areas.  The  suggested  emphasis  would  be  on  providing  solutions  to 
PDSS  issues,  not  simply  airing  these  issues  to  contractors  as  with  the  BTVs.  Suggested  tasks 
for  a  tiger  team  include  tasks  relating  to  speeding  the  widespread  use  of  reusable  software  com¬ 
ponents: 

•  identification  of  reusable  software  components 

•  redevelopment  of  these  components  in  Ada 

•  coordination  of  efforts  dealing  with  reusable  software  components,  including  the  de¬ 
velopment  of  a  standard  DoD  database  of  reusable  software  components  providing  a 
means  to  search  and  extract  from  this  database  based  on  keywords,  abstracts,  or 
specifications  of  the  software  "components"  contained  within  the  library 

•  demonstration  of  benefits  of  standard  software  components  using  Ada 

•  development  of  military  specifications  for  software  components  defining  form,  fit  and 
function,  much  as  is  done  for  hardware  components  today 

4.2.  Communicating  the  Software  Messages 

This  section  attempts  to  define  how  these  messages  can  best  be  communicated  to  industrial 
designers  and  managers  and  government  acquisition  personnel.  As  illustrated  in  Figure  4-1,  the 
software  inventory  of  the  U.S.  Air  Force  is  fairly  diverse.  The  discussions  held  as  part  of  this 
BTV  fact-finding  process  focused  on  *  -  i  issues  relating  primarily  to  weapon  system  software  for 
Air  Force  aircraft. 

The  conclusions  that  emerged  from  the  various  discussions  were  that  the  messages  of  a  general, 
software-oriented  BTV  should  not  be  limited  to  just  radar,  electronic  warfare,  or  any  other  single 


30 


CMU/SEI-87-TR-42 


Air  Fore* 
Software 


Figure  4-1 :  U.S.  Air  Force  Software  Inventory  (adapted  from  AF  and  AFLC  Regulations) 


aspect  of  avionics.  They  should  also  not  be  limited  to  just  one  operational  viewpoint.  The  trip 
should  involve  exposure  to  SAC  and  TAC  as  well  as  the  ALCs.  Many  participants  noted  that 
although  a  general  BTV  on  software  would  be  useful,  a  specific  BTV,  targeted  at  issues  in  any 
one  of  the  categories  of  software  shown  in  Figure  4-1,  would  also  be  extremely  useful  to  an 
industry  audience  familiar  with  that  category  of  software  systems. 

In  general,  software-oriented  personnel  who  met  with  the  fact-finding  team  were  supportive  of  the 
concept  of  a  software-oriented  BTV,  providing  the  BTV  was  oriented  to  show  real  issues  and 
probiems  such  as  poor  documentation  or  ill-defined  interfaces  along  with  effort  necessary  to 
correct  these  problems.  It  was  suggested  that  software-oriented  BTV  participants  get  a  chance  to 
actually  see  fault  isolation  tasks  performed,  and  then  to  visit  the  intermediate  avionics  shops.  It 
was  also  suggested  that  perhaps  a  unit  such  as  the  419th  Tactical  Fighter  Wing,  an  Air  Force 
Reserve  unit,  could  be  used  for  some  of  the  "hands  on"  tours,  because  a  BTV  would  impose  a 
lesser  impact  on  mission  performance  on  this  unit  than  on  an  Active  squadron. 

A  facility  such  as  the  F-16  Avionics  Integration  Support  Facility  (AISF)  could  be  used  to  demon¬ 
strate  the  mission  of  the  ALC.  This  facility  represents  the  expenditure  of  over  thirty  million  dollars 
of  AFSC  computer  funds,  with  over  130  organic  plus  9C  contract  personnel  manning  the  facility. 
Its  purpose  is  to  provide  an  organic  weapon  system  change  capability.  The  AISF  must  provide 
integrated  hardware  and  software  avionics  support  that  is  both  responsive  to  changing  mission 
needs  and  is  cost  effective.  The  role  of  the  AISF  is  illustrated  in  Figure  4-2.  The  AISF  contains 
extensive  computing  resources.  For  the  F-16  A/B,  it  includes  a  DECsystem-10™,  VAX™-1 1/785, 
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and  an  Evans  &  Sutherland  PS  2™.  A  separate  heads-up  display  test  station  includes  a  func¬ 
tional  heads-up  display,  while  the  Dynamic  System  Simulator  includes  a  functional  cockpit 
mockup  with  seat,  sticks  and  displays.  The  F-1 6  C/D  configuration  will  contain  an  IBM®  4381 
central  processor. 


Figure  4-2:  Components  of  a  Typical  Avionics  Integration  Support  Facility  (AISF) 


A  potential  EW  software  BTV  could  travel  to  Eglin  AFB  to  meet  with  F-15  users  and  maintainers 
at  both  the  33rd  Tactical  Fighter  Wing  and  the  Tactical  Air  Warfare  Center,  before  traveling  on  to 
Langley  or  Shaw  as  other  operational  sites.  Seymour  Johnson  was  also  mentioned  as  a  possible 
operational  site,  because  it  has  the  newest  version  of  the  ALQ-131  pod  on  their  F-I6s.  Any  EW 
software  trip  could  conclude  at  Warner  Robins  Air  Logistics  Center  and  the  EWAISF. 

Visits  to  organizations  where  software  use  is  the  primary  means  of  mission  accomplishment,  such 
as  Space  Command  at  Falcon  AFS,  Global  Weather  Central  or  the  AWACS  (E-3)  at  Tinker  AFB, 
were  also  suggested.  The  552nd  Airborne  Warning  and  Control  Wing  at  Tinker  AFB  was  sug¬ 
gested  because  of  their  role  in  aircraft  software  maintenance  and  simulation  for  testing. 
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4.3.  Targeting  the  Message 

This  section  enumerates  the  various  groups  of  personnel  who  may  be  candidates  for  participating 
in  a  software-oriented  BTV.  It  would  be  beneficial  to  involve  both  senior-level  technical  person¬ 
nel,  as  well  as  senior  management,  in  a  software-oriented  BTV.  It  was  also  suggested  that  Air 
Force  program  management  personnel  be  involved  in  this  BTV. 

Key  personnel  who  would  benefit  from  the  exposure  offered  by  a  software-oriented  BTV  are  in 
the  following  categories: 

•  Program  Managers 

•  Directors  or  Managers  of  Software  Engineering 

•  Systems  Engineers 

•  Software  Engineers 

•  Design  Engineers 

•  Logistics  Engineers  (Test  Stations,  Test  Software,  Test  Program  Sets) 

•  Human  Factors  Engineers 

•  Software  Quality  Assurance  Engineers 

•  Systems  Integrators 

•  Acquisition  personnel  from  Air  Force  Systems  Command  and  other  major  commands 

•  System  Program  Office  Directors 

•  System  Program  Office  personnel  (e.g.,  ASD/EN  or  EA) 

•  Operating  personnel,  specifically  pilots  to  expose  them  to  the  realities  of  complex 
software  maintenance  actions 

4.4.  Identifying  the  Software  Blue  Two  Visit 

The  question,  "What  should  the  Software  BTV  be  called?"  is  important  in  communicating  the 
focus  and  target  of  a  software-oriented  BTV  to  decision  makers  in  order  to  attract  an  appropriate 
cadre  of  people  for  the  BTV. 

Several  names  for  software-oriented  BTVs  were  suggested  throughout  the  discussions  at  the 
various  units.  These  include: 

•  Aircraft  Operational  &  Test  Software  BTV 

•  Aircraft  MCCR  Software  BTV 

•  Avionics  Operational  &  Test  Software  BTV 

•  Avionics  PDSS  BTV 

•  Aircraft  Software  Supportability  BTV 

•  Aircraft  Software  Operations  and  Test  Supportability  BTV 

•  Aircraft  Software  "ilities"  BTV 


CMU/SEI-87-TR-42 


33 


•  Aircraft  Software:  Usability  and  Maintainability  BTV 

•  Aircraft  Integrated  Software  BTV 

•  Aircraft  Integrated  Software  Support  BTV 

•  Integrated  Aircraft  Software  Support  BTV 

•  PDSC  BTV 

•  Software  Support  BTV 

•  Electronic  Warfare  Software  BTV 

•  Software  (topic)  BTV  -  with  "topic"  being  your  choice  of:  avionics,  EW,  etc. 

A  consensus  was  reached  that  the  best  name  for  this  BTV  was  Aircraft  Operational  and  Test 
Software  BTV. 
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5.  Conclusion 


The  fact-finding  team,  based  on  information  gathered  from  discussions,  has  attempted  to  sum¬ 
marize  from  a  maintainer’s  perspective,  the  issues  which  ultimately  decide  supportability  of  soft¬ 
ware  for  a  weapon  system.  As  reported,  these  issues  play  a  major  role  in  determining  the  quality 
of  maintenance  and  life  cy^'e  costs: 

•  weapon  system  orientation 

•  acquisition 

•  software  development 

•  post  deployment  software  support 

•  personnel  management 

These  findings,  although  specifically  intended  to  define  the  scope,  message,  and  target  audience 
for  a  software-oriented  BTV,  should  also  serve  as  indicators  to  Air  Force  program  managers  in 
understanding  the  approaches  that  must  be  taken  to  improve  the  current  acquisition  and  support 
processes. 

From  the  activities  visited,  it  is  apparent  that  there  are  messages  from  software  maintained  and 
users  to  industry  and  government,  and  that  software  with  its  associated  reliability  and  maintain¬ 
ability  concerns  is  a  candidate  for  BTV  exposure.  It  is  the  team’s  consensus  that  these  concerns 
within  each  of  the  Air  Force's  software  resources  (shown  iri  Figure  4-1)  are  deserving  of  manage¬ 
ment  attention  through  one  platform  or  another.  However,  the  BTV  priority  should  be  given  to 
those  resources  with  the  greatest  impact  on  combat  capability  and  national  defense,  such  as 
critical  weapon  system  resources.  Problems  in  supporting  these  systems  degrades  the  ability  of 
the  Air  Force  to  meet  its  mission;  thus,  a  "show  and  teH"  platform  is  key  in  elevating  the  software 
maintainer’s  concerns  to  the  decision  maker  level. 

Before  conducting  a  software-oriented  BTV,  there  are  some  distinctions  between  the  hardware 
and  software  environments  that  must  be  understood.  This  is  due,  for  the  most  part,  to  the  ob¬ 
vious  differences  in  the  nature  of  the  two;  hardware  deals  with  visible,  tangible  entities,  and 
software  addresses  logical  structures  and  complex  processes. 

However,  the  most  significant  distinction  noted  by  the  team  in  terms  of  presenting  software  within 
the  BTV  platform  is  the  contrast  of  constraints  underlying  maintenance  of  hardware  and  software. 
The  flight  line  environment  typical  of  a  BTV  serves  to  communicate  the  realistic  conditions  under 
which  the  maintenance  of  hardware  must  be  performed.  The  cold  weather,  the  open  hanger,  the 
lack  of  spare  parts,  and  the  use  of  special  support  gear  effectively  highlight  to  industry  the  day-to- 
day  constraints  of  maintenance  and  the  necessity  for  future  reliable  and  maintainable  weapon 
system  components  and  support  structures.  In  the  software  environment,  the  "constrains"  are 
largely  due  to  the  combination  of  technical  and  management  variables  which  are  in  place  as  a 
result  of  the  system  —  the  way  the  Air  Force  currently  does  business.  These  factors  can  be 
changed.  The  fact  that  the  ALCs  do  not  have  adequate  computer  support  and  support  tools  is 
not  a  "real  world"  limitation  like  the  freezing  weather  in  Minot,  ND,  or  maintaining  an  aircraft  while 
in  chemical/biological/radiological  gear.  Thus,  care  must  be  taken  not  to  present  to  industry  a 
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look  at  unrealistic  limitations;  and  although  these  variables  are  valid  in  describing  the  maintainer’s 
inability  to  perform  the  job,  one  cannot  assume  that  the  same  limitations  apply  as  with  hardware. 


The  intersection  of  the  two  environments  centers  around  two  main  issues: 

•  the  impact  of  a  hardware  or  software  design  on  maintainability 

•  inability  to  adequately  support  these  systems  can  determine  the  success  or  failure  of 
the  Air  Force  mission 

As  the  first  in  a  series  of  Blue  Two  Visits  on  weapon  system  software  support,  it  is  recommended 
that  the  visit  focus  on  BIT/ATE  or  EW  software.  These  subjects  take  advantage  of  the  overlap  of 
environments  and  will  make  it  easier  to  transition  the  target  audience  from  what  appears  to  be 
purely  hardware  to  the  ultimate  software  driver  and  underlying  support  issues.  It  is  these  issues 
that  are  the  concerns  of  the  software  maintainers.  In  the  BIT/ATE  arena,  the  lack  of  sufficient  test 
points  translates  to  more  CNDs,  more  spares  on  hand,  and  more  cost  to  the  government.  In  the 
EW  arena,  the  inability  to  do  rapid  reprogramming  is  costing  the  government  in  terms  of  weapon 
system  readiness.  If  not  properly  focussed  upon,  this  issue  alone  could  present  a  voluntary 
choke  point  in  a  wartime  theater. 

For  BIT/ATE  concerns,  Dyess  AFB  (96th  BMW),  Hill  AFB  (388th  TFW/OO-ALC),  and  Robins  AFB 
(WR-ALC/MAI)  are  all  candidate  locations.  For  a  software  BTV  oriented  at  EW  concerns,  Robins 
AFB  (WR-ALC/MMR),  Eglin  AFB  (TAWC),  and  Shaw  AFB  (363rd  TFW/MA)  are  recommended  as 
candidate  locations. 

In  terms  of  the  target  audience,  attentions  should  particularly  be  given  to  the  Air  Force  acquisition 
community.  As  participants  repeatedly  pointed  out,  many  problems  appear  to  stem  from: 

•  a  lack  of  "life  cycle"  consciousness  on  the  part  of  government  program  managers 

•  a  failing  of  system  program  offices  to  fully  assess  the  importance  of  software  support 

•  a  lack  of  understanding  of  the  impact  software  has  on  combat  capability  and  weapon 
system  readiness. 

To  achieve  the  greatest  benefit  with  the  software  BTVs,  participation  must  include  representatives 
of  major  system  acquisitions. 

Lastly,  to  minimize  the  impact  on  planned  BTV  efforts  described  in  Section  1 ,  it  is  recommended 
that  the  software  BTV  be  conducted  in  May  1988.  This  is  necessary  to  allow  for  adequate 
pre-site  planning  needed  to  ensure  effective  "hands-on"  demonstrations  and  dialogue. 
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Appendix  A:  List  of  Acronyms  and  Office  Symbols 


AAC 

AF 

AFALC 

AFB 

AFCOLR 

/MEI 

/XRI 

AFLC 

AFOTEC 

AFRES 

AFS 

AFSC 

AGS 

Al 

AIS 

AISF 

ALC 

ARC 

ASD 

/RWA 


ATE 

ATF 

AWACS 

BIT 

BMW 

BTV 

C3I 

CALS 

CAMS 

CASE 

CEPS 

CITS 

CND 

CRS 

DID 

DO 

DoD 

DoD-STD 

DPML 

DPMS 

EW 

EWAISF 

FOT&E 

GIMADS 

HQ 

HUD 

ICEWMD 

IGES 

IMA 


Alaskan  Air  Command 
Air  Force 

Air  Force  Acquisition  Logistics  Center 
Air  Force  Base 

Air  Force  Coordinating  Office  for  Logistics  Research 

HQ  USAF,  AFCOLR,  Maintenance  and  Engineering  Division,  Independent 

Research  &  Development,  Blue  Two  Visit  Program  (office  symbol) 

HQ  USAF,  AFCOLR,  Plans  &  Programs  Division,  Information  Systems 
Branch  (office  symbol) 

Air  Force  Logistics  Command 

Air  Force  Operational  Test  and  Evaluation  Center 

Air  Force  Reserve 

Air  Force  Station 

Air  Force  Specialty  Code,  Air  Force  Systems  Command 

Aircraft  Generation  Squadron 

Artificial  intelligence 

Avionics  Intermediate  Shop 

Avionics  Integration  Support  Facility 

Air  Logistics  Center 

Area  Reprogramming  Capability 

Aeronautical  Systems  Division 

Inter-Command  Electronic  Warfare  Management  Directorate  (ICEWMD) 
(functional  office  symbol) 

Automatic  Test  Equipment 

Advanced  Tactical  Fighter 

Airborne  Warning  and  Control  System 

Built  In  Test 

Bombardment  Wing 

Blue  Two  Visit 

Command,  control,  communications  and  intelligence 

Computer-Aided  Acquisition  and  Logistics  Support 

Core  Automated  Maintenance  System 

Computer  Aided  Software  Engineering 

CITS  Expert  Parameter  System 

Central  Integrated  Test  System 

Could  Not  Duplicate 

Component  Repair  Squadron 

Data  Item  Description 

Deputy  for  Operations  (office  symbol) 

Department  of  Defense 
Department  of  Defense  Standard 
Deputy  Program  Manager  for  Logistics 
Deputy  Program  Manager  for  Software 
Electronic  Warfare 

Electronic  Warfare  Avionics  Integration  Support  Facility 
Follow-on  Operational  Test  and  Evaluation 
Generic  Integrated  Maintenance  Diagnostics  System 
Headquarters 
Heads  Up  Display 

Inter-Command  Electronic  Warfare  Management  Directorate 
Initial  Graphic  Exchange  Specification 
Individual  Mobilization  Augmentee 
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iv&v 

LG 

LGMMD 

LRU 

MACIA 

MAJCOM 

MAI 

MAIT 

MAITB 


MAITBC 

MAITC 

MAITCA 

MAKTF 

MAMC 

MAP 

MATE 

MCCR 

MEI 

MMEC 

MMEC 

MM  EC  A 


MMECR 

MMECT 

MIL-STD 

MMR 

MMRR 

MMRRA 


MMRRC 


MMRRIA 


Independent  Validation  and  Verification 
Deputy  for  Logistics  ( office  symbol) 

Logistics  Maintenance  Management  Data  Branch,  HQ  SAC  ( office  symbol) 
Line  Replaceable  Unit 

Integrated  Avionics  Branch,  Component  Repair  Squadron  (office  symbol) 
Major  Command 

Airborne  Electronics  Division,  Directorate  of  Maintenance  (WR-ALC  office 
symbol) 

Software  Support  Center  Branch,  Airborne  Electronics  Division,  Directorate 
of  Maintenance  (WR-ALC  office  symbol) 

Radar  and  Space  Communications  Section,  Software  Support  Center 
Branch,  Airborne  Electronics  Division,  Directorate  of  Maintenance  (WR-ALC 
office  symbol) 

Countermeasures  Unit,  Radar  and  Space  Communications  Section, 
Software  Support  Center  Branch,  Airborne  Electronics  Division,  Directorate 
of  Maintenance  (WR-ALC  office  symbol) 

Tactical  Support  Section,  Software  Support  Center  Branch,  Airborne 
Electronics  Division,  Directorate  of  Maintenance  (WR-ALC  office  symbol) 
F-15  Radar  Support  Unit,  Tactical  Support  Section,  Software  Support 
Center  Branch,  Airborne  Electronics  Division,  Directorate  of  Maintenance 
( WR-ALC  office  symbol) 

Software  Support  Center  Branch,  Missile  and  Aircraft  Systems  Division, 
Directorate  of  Maintenance  (OO-ALC  office  symbol) 

CITS  Maturation  Branch,  Maintenance  Control  Division,  Deputy 
Commander  for  Maintenance  (office  symbol) 

Maintenance  Systems  Analysis,  Maintenance  Control  Division,  Deputy 
Commander  for  Maintenance  (office  symbol) 

Modular  Automatic  Test  Equipment 
Mission  Critical  Computer  Resources 

HQ  USAF,  AFCOLR,  Maintenance  and  Engineering  Division,  Independent 
Research  &  Development,  Blue  Two  Visit  Program  (office  symbol) 

Aircraft  Computer  Resources  Branch,  Engineering  Division,  Directorate  of 
Material  Management  (OO-ALC  office  symbol) 

Integration  Support  Facility  Operations  Branch,  Systems  Engineering 
Division,  Directorate  of  Material  Management  ( WR-ALC  office  symbol) 

F-16  Fire  Control/HUD  OFP  Development  Section,  Aircraft  Computer 
Resources  Branch,  Engineering  Division,  Directorate  of  Material 
Management  (OO-ALC  office  symbol) 

F-16  Radar/Stores  OF P  Development  Section,  Aircraft  Computer 
Resources  Branch,  Engineering  Division,  Directorate  of  Material 
Management  (OO-ALC  office  symbol) 

F-4  AISF  Section,  Aircraft  Computer  Resources  Branch,  Engineering 
Division,  Directorate  of  Material  Management  (OO-ALC  office  symbol) 
Military  Standard 

Electronic  Warfare  (EW)  Management  Division,  Directorate  of  Material 
Management  (WR-ALC  office  symbol) 

Engineering  and  Reliability  Branch,  Electronic  Warfare  (EW)  Management 
Division,  Directorate  of  Material  Management  (WR-ALC  office  symbol) 
Emergency  Reprogramming  Center,  Simulation  and  Evaluation  Section, 
Engineering  and  Reliability  Branch,  Electronic  Warfare  (EW)  Management 
Division,  Directorate  of  Material  Management  ( WR-ALC  office  symbol) 

EW  Active  Systems  Section,  Engineering  and  Reliability  Branch,  Electronic 
Warfare  (EW)  Management  Division,  Directorate  of  Material  Management 
( WR-ALC  office  symbol) 

APR-38  Unit,  EW  Integrated  Systems  Section,  Engineering  and  Reliability 
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Branch,  Electronic  Warfare  (EW)  Management  Division,  Directorate  of 
Material  Management  ( WR-ALC  office  symbol) 

MMRRIC  EF-11 1/ALQ-125  Unit,  EW  Integrated  Systems  Section,  Engineering  and 
Reliability  Branch,  Electronic  Warfare  (EW)  Management  Division, 
Directorate  of  Material  Management  ( WR-ALC  office  symbol) 

MMRRV  Radar  Warning  Receiver  Section,  Engineering  and  Reliability  Branch, 
Electronic  Warfare  (EW)  Management  Division,  Directorate  of  Material 
Management  (WR-ALC  office  symbol) 

MMRRVE  ALR-74/F-15  TEWS  Unit,  Radar  Warning  Receiver  Section,  Engineering 
and  Reliability  Branch,  Electronic  Warfare  (EW)  Management  Division, 
Directorate  of  Material  Management  (WR-ALC  office  symbol) 

NCO  Non-commissioned  officer 

NCOIC  Non-commissioned  officer  in  charge 

OFP  Operational  Flight  Program 

OGP  Operational  Ground  Program 

OO-ALC  Ogden  Air  Logistics  Center 

/MAKTF  Software  Support  Center  Branch,  Missile  and  Aircraft  Systems  Division, 
Directorate  of  Maintenance  (OO-ALC  office  symbol) 

/MMEC  Aircraft  Computer  Resources  Branch,  Engineering  Division,  Directorate  of 
Material  Management  (OO-ALC  office  symbol) 

/MMECA  F-16  Fire  Control/HUD  OFP  Development  Section,  Aircraft  Computer 
Resources  Branch,  Engineering  Division,  Directorate  of  Material 
Management  (OO-ALC  office  symbol) 

/MMECR  F-16  Radar/Stores  OFP  Development  Section,  Aircraft  Computer 
Resources  Branch,  Engineering  Division,  Directorate  of  Material 
Management  (OO-ALC  office  symbol) 

/MMECT  F-4  AISF  Section,  Aircraft  Computer  Resources  Branch,  Engineering 
Division,  Directorate  of  Material  Management  (OO-ALC  office  symbol) 

OTP  Operational  Test  Program 

PACAF  Pacific  Air  Forces 

PDSS  Post  Deployment  Software  Support 

PMD  Program  Management  Directive 

PROM  Programmable  read-only  memory 

RF  Radio  Frequency 

RWA  Inter-Command  Electronic  Warfare  Management  Directorate  (ICEWMD), 

Aeronautical  Systems  Division  ( functional  office  symbol) 

SAC  Strategic  Air  Command 

/LG  Deputy  for  Logistics  (office  symbol) 

/LGMMD  Logistics  Maintenance  Management  Data  Branch  ( office  symbol) 

SEI  Software  Engineering  Institute 

SGML  Standard  Generalized  Markup  Language 

SMDPS  Strategic  Mission  Data  Planning  System 

SPM  System  Program  Manager 

SPO  Systems  Program  Office 

SRU  Shop  Replaceable  Unit 

SSC  Software  Support  Center 

SSE  Software  Support  Environment 

TAC  Tactical  Air  Command 

TAWC  Tactical  Air  Warfare  Center 

TDLA  Maintenance  Evaluator  for  Avionics  Maintenance  Squadron  (office  symbol) 

TDS  Deputy  for  Software  (office  symbol) 

TFW  Tactical  Fighter  Wing 

TODO  Technical  Order  Distribution  Officer 

TPS  Test  Program  Set 

USAF  United  States  Air  Force 
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USAFE  United  States  Air  Forces  in  Europe 

USAFR  United  States  Air  Force  Reserve  (see  also  AFRES) 

UUT  Unit  under  test 

WR-ALC  Warner  Robins  Air  Logistics  Center 

/MAI  Airborne  Electronics  Division,  Directorate  of  Maintenance  ( WR-ALC  office 
symbol) 

/MAIT  Software  Support  Center  Branch,  Airborne  Electronics  Division,  Directorate 
of  Maintenance  ( WR-ALC  office  symbol) 

/MAITB  Radar  and  Space  Communications  Section,  Software  Support  Center 

Branch,  Airborne  Electronics  Division,  Directorate  of  Maintenance  (WR-ALC 
office  symbol) 

/MAITBC  Countermeasures  Unit,  Radar  and  Space  Communications  Section, 

Software  Support  Center  Branch,  Airborne  Electronics  Division,  Directorate 
of  Maintenance  ( WR-ALC  office  symbol) 

/MAITC  Tactical  Support  Section,  Software  Support  Center  Branch,  Airborne 

Electronics  Division,  Directorate  of  Maintenance  (WR-ALC  office  symbol) 
/MAITCA  F-15  Radar  Support  Unit,  Tactical  Support  Section,  Software  Support 

Center  Branch,  Airborne  Electronics  Division,  Directorate  of  Maintenance 
( WR-ALC  office  symbol) 

/MM EC  Integration  Support  Facility  Operations  Branch,  Systems  Engineering 
Division,  Directorate  of  Material  Management  (WR-ALC  office  symbol) 

/MMR  Electronic  Warfare  (EW)  Management  Division,  Directorate  of  Material 
Management  (WR-ALC  office  symbol) 

/MMRR  Engineering  and  Reliability  Branch,  Electronic  Warfare  (EW)  Management 
Division,  Directorate  of  Material  Management  (WR-ALC  office  symbol) 
/MMRRA  Emergency  Reprogramming  Center,  Simulation  and  Evaluation  Section, 
Engineering  and  Reliability  Branch,  Electronic  Warfare  (EW)  Management 
Division,  Directorate  of  Material  Management  ( WR-ALC  office  symbol) 
/MMRRC  EW  Active  Systems  Section,  Engineering  and  Reliability  Branch,  Electronic 
Warfare  (EW)  Management  Division,  Directorate  of  Material  Management 
( WR-ALC  office  symbol) 

/MMRRIA  APR-38  Unit,  EW  Integrated  Systems  Section,  Engineering  and  Reliability 
Branch,  Electronic  Warfare  (EW)  Management  Division,  Directorate  of 
Material  Management  (WR-ALC  office  symbol) 

/MMRRICEF-111/ALG-125  Unit,  EW  Integrated  Systems  Section,  Engineering  and 
Reliability  Branch,  Electronic  Warfare  (EW)  Management  Division, 
Directorate  of  Material  Management  (WR-ALC  office  symbol) 

/MMRRV  Radar  Warning  Receiver  Section,  Engineering  and  Reliability  Branch, 
Electronic  Warfare  (EW)  Management  Division,  Directorate  of  Material 
Management  (WR-ALC  office  symbol) 

/MMRRVE 

ALR-74/F-15  TEWS  Unit,  Radar  Warning  Receiver  Section,  Engineering 
and  Reliability  Branch,  Electronic  Warfare  (EW)  Management  Division, 
Directorate  of  Material  Management  (WR-ALC  office  symbol) 

XRI  HQ  USAF,  AFCOLR,  Plans  &  Programs  Division,  Information  Systems 

Branch  (office  symbol) 
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Appendix  B:  Software  Blue  Two  Visit  Fact-Finding 


Participants 

Air  Force  Coordinating  Office 
for  Logistics  Research 
(HQ  USAF) 

Capt.  Benita  L.  Gilliard 
Air  Force  Coordinating  Office 
for  Logistics  Research 
(HQ  USAF) 

AFCOLR/XRI 

Wright-Patterson  AFB,  OH  45433-5000 
Office  Phone:  513-255-4758 
AV  785-4758 

CMSgt.  Charles  M.  Worm 
Air  Force  Coordinating  Office 
for  Logistics  Research 
(HQ  USAF) 

AFCOLR/MEI 

Wright-Patterson  AFB,  OH  45433-5000 
Office  Phone:  513-255-4758 
AV  785-4758 


Aeronautical  Systems  Division 

Landon  O.  Sager 
Inter-Command  Electronic 
Warfare  Management  Directorate, 
Aeronautical  Systems  Division 
ASD/RWA 

Wright-Patterson  AFB,  OH  45433-5000 
Office  Phone:  513-255-2108 
AV  785-2108 


Software  Engineering  Institute 

William  E.  Hefley 
Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 
Office  Phone:  412-268-7793 


Ogden  Air  Logistics  Center 

Rick  Holsman 

Ogden  Air  Logistics  Center 

OO-ALC/MMEC 

Hill  AFB,  UT  84056-5000 

Office  Phone:  801-777-7355 

AV  458-7355 

Major  Gary  S.  Hochstetler,  USAFR 
Ogden  Air  Logistics  Center 
OO-ALC/MMEC 
Hili  AFB,  UT  84056-5000 

Leon  Oldham 

Ogden  Air  Logistics  Center 
OO-ALC/MMECR 
Hill  AFB,  UT  84056-5000 
Office  Phone:  801-777-7232 
AV  458-7232 

Bruce  W.  Rudd 

Ogden  Air  Looistics  Center 

OO-ALC/MMEC  A 

Hill  AFB,  UT  84056-5000 

Office  Phone:  801-777-7336 

AV  458-7336 

Robert  Sharp 

Ogden  Air  Logistics  Center 
OO-ALC/MAKTF 
Hill  AFB,  UT  84056-5000 
Office  Phone:  801-777-7855 
AV  458-7855 

Bill  Frost 

Ogden  Air  Logistics  Center 

OO-ALC/MMECT 

Hill  AFB,  UT  84056-5000 


388,h  Tactical  Fighter  Wing 

TSgt.  Jeffrey  A.  Carey 
388  CRS/MACIA 
Hill  AFB,  UT  84056-5000 
Office  Phone:  801-777-5494 
AV  458-5494 

TSgt.  Rudulph  W.  Peart 
388  CRS/MACIA 
Hill  AFB,  UT  84056-5000 
Office  Phone:  801-777-5494 
AV  458-5494 

SSgt.  Cht  stopher  B.  Gifford 
388  AGS/MAABC 
Hill  AFB,  UT  84056-5000 
Office  Phone:  801-777-3448 
AV  458-3448 


Strategic  Air  Command 

Capt.  Danie)  R.  Bliss 
Headquarters,  Strategic  Air  Command 
HQ  SAC/LGMMD 
Offutt  AFB,  NE  68113 
Office  Phone: 

AV  271-5627 


96th  Bombardment  Wing  (Heavy) 

Capt.  Steve  Hackett 
96  BMW/MAMC 
Dyess  AFB,  TX  79607 
Office  Phone:  915-696-2653 
AV  461-2653 

CMSgt.  Robert  Lewis 
96  BMW/MAP 
Dyess  AFB,  TX  79607 


Air  Force  Operational  Test  &  Evaluation 

Major  Gary  F.  Giesecke 
AFOTEC  B-1B  FOT  &  E/TDS 
Dyess  AFB,  TX  79607 
Office  Phone:  915-696-4436 
AV  461-4436 


Capt.  Glenn  Tuley 
AFOTEC  B-1B  FOT  &  E/TDS 
Dyess  AFB,  TX  79607 
Office  Phone:  915-696-4428 
AV  461-4428 

1  Lt.  Rich  Dale 

AFOTEC  B-1B  FOT  &  E/TDS 
Dyess  AFB,  TX  79607 
Office  Phone:  915-696-4424 
AV  461-4424 

2Lt.  Emily  Andrew 
AFOTEC  B-1B  FOT  &  E/TDS 
Dyess  AFB,  TX  79607 
Office  Phone:  915-696-4433 
AV  461-4433 

MSgt  Mel  Noble 
AFOTEC  B-1B  FOT  &  E/TDLA 
Dyess  AFB,  TX  79607 
Office  Phone:  915-696-3182 
AV  461-3182 

SSgt.  W.  R.  Wanamaker 
AFOTEC  B-1B  FOT  &  E/TDLA 
Dyess  AFB,  TX  79607 
Office  Phone:  915-696-4411 
AV  461-4411 


Warner  Robins  Air  Logistics  Center 

JohnJ.  LaVecchia 

Warner  Robins  Air  Logistics  Center 

WR-ALC/MMRR 

Robins  AFB,  GA  31098-5149 

Office  Phone:  912-926-5948 

AV  468-5948 

Harry  Jennings 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MMRRA 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-4611 
AV  468-4611 

Dorothy  Jackson 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MMRRIC 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-4926 
AV  468-4926 
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Joseph  Watwood 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MMRRIA 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-4617 
AV  468-4617 


Michael  S.  Tipton 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MMRRC 
Robins  AFB,  G A  31098-5149 
Office  Phone:  912-926-4525 
AV  468-4525 

William  Calkins 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MMRRV 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-4896 
AV  468-4896 

Roy  P.  Oliver 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MMRRVE 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-2588 
AV  468-2588 

William  Raymond  Haggard 
Warner  Robins  Air  Logistics  Center 
WR-ALC/MMEC 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-3934 
AV  468-3934 

Larry  K.  Israel 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MAIT 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-2457 
AV  468-2457 

Lonnie  Y.  Totty 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MAITB 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-4725 
AV  468-4725 


Jerry  L.  Watts 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MAITBC 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-3207 
AV  468-3207 

Charles  R.  Singleton 

Warner  Robins  Air  Logistics  Center 

WR-ALC/MAITC 

Robins  AFB,  GA  31098-5149 

Office  Phone:  912-926-5061 

AV  468-5061 

Jim  Mosely 

Warner  Robins  Air  Logistics  Center 
WR-ALC/MAITCA 
Robins  AFB,  GA  31098-5149 
Office  Phone:  912-926-5061 
AV  468-5061 
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Appendix  C:  Ogden  Air  Logistics  Center  Briefing 
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